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GOODNESS OF FIT OF TREND CURVES AND SIGNIFICANCE 
OF TREND DIFFERENCES 


E. F. LINDQUIST 


STATE UNIVERSITY OF IOWA 


Experimental studies of the successive changes gp agg rep- 
resented by curves describing laws of learning and other similar 
functional relationships) in a criterion variable accompanying ex- 
perimental variations in a given “treatment,” and experimental com- 
parisons of such changes for different populations or for different 
treatments, constitute a large and important class of psychological 
experiments. In most such experiments, no attempt has been made to 
analyze or to make allowance for errors of sampling or of observa- 
ition. In many others, the techniques of error analysis that have been 
employed have been inefficient, inexact, or inappropriate. This paper 
suggests tests, using the methods of analysis of variance, of certain 
hypotheses concerning trends and trend differences in sample means 
obtained in experiments of this general type. For means of succes- 
sive independent samples, tests are provided of the hypotheses: (H,) 


that there is no trend, or that the trend is a horizontal straight line, 
(H,) that there is a linear trend ,(H,) that the trend is as described 


by a line not derived from the observed means, and (H,) that the 


trend is as described by a line fitted to the observed means. Tests 
are also provided of similar hypotheses (H,, H,, H,, and H,, re- 


spectively) for means of successive measurements of the same sam- 
ple. Finally, tests are provided of the null hypotheses that there is 
no difference in trend in two series of means: (H,) when each mean 


in each series is based on an independent sample, (H,,) when each 


pair of corresponding means is based on an independent sample, 
(H,,) when each series of means is based on an independent sam- 


ple, and (H,.) when both series are based on a single sample. 


Case 1: Test for Trend in Means Based on Independent Samples 


Suppose that several groups of subjects were originally selected 
at random from the same population and that one group was sub- 
jected to a certain treatment for one hour, another group was given 
the same treatment for two hours, another for three, etc. For each 
group, measures of a certain trait (Y) were obtained immediately 
following administration of the treatment. Figure 1 represents the 
possible outcome of an experiment of this type, the open dots repre- 
senting the means of the criterion variable for the various groups 
arranged in order of duration (X) of treatment. The problem would 
of course be the same if X represented amount, or intensity, or num- 
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Experimental Variable (X) 


Figure 1 
ber of repetitions, etc., of the treatment. (The X increment or inter- 
val need not be uniform.) 


The purpose of the experiment is to test the null hypothesis (H,) 
that the criterion variable (Y) is unaffected by increases in X or 
that the line of population means is a horizontal straight line. The 
test needed is obviously that provided by the method of simple analy- 
ses of variance. 


Procedure: Assume that the criterion variable is normally dis- 
tributed and has the same variance for all populations, i.e., for all 
values of X. Let N represent the total number of cases in all sam- 
ples (groups). 


Using the methods of simple analyses of variance, compute the 
“between groups” or “between intervals” variance (07) and the 


“within groups” variance (07). Under the hypothesis, each is an un- 


biased estimate of the commom population variance. The hypothesis 
may then be tested by 


F=ot/a2 
for which the degrees of freedom are (k — 1) and (N— k). 


Note: Careful consideration should be given in this and the follow- 
ing tests, to the assumption that the population variance (of Y) is 
the same for all values of X. This is sure to be an unsound assump- 
tion if the line of means begins at, or at any point closely approaches, 
the base line (Y = 0) and if negative values of Y are impossible. In 
such cases, however, it may sometimes be possible to test the hypothe- 
sis along only that part of the X-scale for which the Y-variance may 
be considered fairly constant, disregarding the rest of the data. 











eY- 


I;) 


‘he 
ly- 


is- 
all 


he 


he 


‘is 


IS 


Tra a 


<¢ 














E. F. LINDQUIST 67 


Case 2: Test for Trend in Means Based on a Single Sample 


The experimental design is similar to that in Case 1, except that 
only one sample of subjects is employed—k successive measures of the 
criterion, corresponding to the successive values of X , being obtained 
for each subject. The purpose in this case is to test the hypothesis 
(H.) that, for the population involved, the increases in X are not ac- 
companied by any changes in the Y-means, or that the regression line 
of population means is a horizontal straight line. Figure 1 might 
again represent the experimental results, except that in this case all 
of the means are based on the same subjects. 

Procedure: Let n represent the number of subjects, let Y; rep- 
resent any measure of the criterion variable for the i-th value of X , 


_and let M; represent the mean of the Y;’s. 


Consider the individual Y-measures as arranged in a table in 
which the k columns correspond to the various values of X , and the 
m rows correspond to the individual subjects. Using the methods of 
analysis of variance, compute the “between columns” or “between in- 
tervals” variance (7) and the “intervals < subjects interaction” 


variance (07 .) for this table. The hypothesis may then be tested by 
ites ‘ie 


for which the degrees of freedom are (kK — 1) and (k —1)( n—1). 

This test involves the important assumptions that, for the en- 
tire population sampled, the regression lines for all subjects have 
zero slope, or are all parallel, and that deviations from individual 
linear regression are normally distributed and of equal variance for 
all indidviduals. Under these assumptions, both o and o are un- 


biased estimates of this common variance if the hypothesis is true. 


Case 3: Test for Linearity of Means Based on Independent Samples 


In an experiment designed like that in Case 1, it may be desired 
to test the hypothesis (H;) that increases in X are accompanied by 
proportional changes in Y, or that there is a linear relationship be- 
tween X and Y. The hypothetical line of population means in this 


case is a straight line, but neither its slope nor its Y-intercept is - 


specified. 


Procedure: Assume that Y is normally distributed and has the 
same variance for each value of X. 


Consider the data as arranged in a correlation table, in which 
the k columns correspond to the various samples, or to the X-inter- 
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vals, and the rows correspond to the Y-intervals. Compute the “with- 
in columns” variance (0° ) for the criterion variable. Compute also 


the “sum of squares” for “between columns” or “between intervals” 
(ss). Then, using the methods of analysis of co-variance,* compute 


the total “sum of products” (Szy), and the total “sum of squares” 
(>?) for the X-distribution. 


The hypothesis may now be tested} by 


eg — (2y)’ 7 


2 
F = =a / ow > 
k—2 
for which the degrees of freedom are (k — 2) and (N — k). 
Since the null hypothesis tested in Case I represents a special 
case of linear relationship, there would be little point here in testing 
for linearity until the null hypothesis had first been proved untenable. 








Case 4: Test for Linearity of Means Based on a Single Sample 


The experimental design is like that in Case 2, k successive meas- 
ures of Y being obtained for each individual in a single sample. The 
purpose is to test the hypothesis (H,) that there is a linear relation- 
ship between X and Y. (See Case 3.) 


Procedure: Compute the “interaction” variance (a * as in Case 
x 
2. Compute also the “sum of squares” for “between intervals” (ss;). 


Using the methods of analysis of co-variance,t compute the total 
“sum of products” (Sy), and the total “sum of squares” (Sx?) for 
the X-distribution. 


The hypothesis may now be tested by 


(Sey)? / 





S68. > : 
x 
ies k—2 f 3 


for which the degrees of freedom are (k — 2) and (n—1)(k—1). 
This test involves the assumptions that all individual regression 
lines are linear and parallel,§ and that deviations from individual re- 





* Lindquist, E. F. ee analysis in educationa] research, Boston: 
Houghton-Mifflin, 1940, pp. 180-19 


+ Ibid., pp. 235-238. 
t Ibid., pp. 180-186, 196-203, 235-238. 


§ This article (somewhat differently agen i , was first used in lithoprinted 
form in the author’s classes in March, 1945, and is now being published in re- 
sponse to numerous requests that it be made more generally available in perma- 
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gression are normally distributed and of equal variance for all sub- 
jects. 


Case 5: Goodness of Fit of Curves Not Derived from the 
Experimental Data (Independent Samples) 


In an experiment designed like that in Case 1, it may be desired 
to test an a priori hypothesis which specifies the exact value* of the 
population mean corresponding to each value of X , or which com- 
pletely defines the curve of population means (L in Fig. 2). 

This problem may be stated in general terms as follows: Given 
a random sample from each of k populations, to test the hypothesis 
(H,;) that the successive population means* have the values My, -:- ; 
My; ---, and My, , respectively. 


Procedure: Let Y; represent a single measure in the i-th sample, 
let n; be the number of cases in the i-th sample, and let N = Sn;. 
The total sum of squared deviations of the individual measures from 
their respective hypothetical means may then be analyzed into two 
independent components, as follows: 


DD (Vi — Mui)? = Si (Mi — Mui)? + & D(¥i — Mi)?. 


The first component is computed by finding the weighted sum of 
squared deviations of the sample means from their respective hypo- 
thetical means and may be called the “departure from hypothesis” 
sum of squares. The second is the sum of squares for “within groups” 
and is computed as a residual in simple analysis of variance (as in 
Case 1). The hypothesis may now be tested by means of 


Dni(Mi— Mui)? / > S(Vi—M;)? 
k N-k ~ 


Since the hypothesis is not derived from the experimental data 
in any way, there is no loss of degrees of freedom in the variance esti- 





r= 


*It does not matter how one arrives at these values, so long as they are not 
derived from the experimental data. They may be separately and arbitrarily set, 
or derived from a regression formula, or read from a graph which has been fitted 
fe Ry se: ra to the means derived in a previous experiment, or obtained in any 
other fashion. 


- 





nent form. H. W. Alexander has since published an article on “A General Test 
for Trend,” Psychological Bulletin, 1946, 48, 583-557, which contains tests 
for the hypotheses here designated as H, and H,. Alexander’s tests, while con- 


siderably more complex, are superior in that they provide for the possibility of 
individual differences in regression. Where there is no good reason to suspect 
— regression, the simpler tests here suggested may perhaps be safely 
employed. 
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mate based on the “departure from hypothesis” sum of squares. Hence, 
the d. f. for this F are k and (N — k). (The denominator is the oO of 


Case 1.) 

This test of H; has the serious weakness that it does not take ade- 
quately into consideration either the direction or the pattern of the dis- 
crepancies. In Figure 2, for example, in which d, = d’, and d; = d’;;, 
this procedure would have the same outcome if the sample means fell at 


Criterion Measure (Y) 











Experimental Variable (X) 


Figure 2 


M,, M.,M,;, and M, as if they fell at M,, M’., M’,;, and M,. How- 
ever, it obviously is more difficult to account for M,, M., M;, and M, 
jointly as chance deviations from line L than it is to so account for 
M,, M’., M’;, and M,, since, if L is a true hypothesis, one would ex- 
pect some of the sample means to fall on either side of L, instead of 
all on one side. On the other hand, the pattern formed by M,, M., 
M;, and M, is much more consistent with the form of the line L than 
is the pattern formed by M,, M’., M’;, and M,. If L were shifted 
upward to the position L’, for instance, it might represent a very ten- 
able hypothesis as far as M,, M., M;, and M, are concerned. (In 
other words, if the hypothesis (H,) is of the form Y = f(x) + C, it 
is possible that the f(x) correctly describes the pattern of the popu- 
lation means, but that the vertical placement of the pattern is not 
correctly given by C.) 

It is thus evident that the F-test suggested above might lead to 
the rejection of an hypothesis which is very acceptable as far as pat- 
tern alone is concerned. What is needed, then, is a separate test con- 
cerned only with the hypothesis (H,,) that the pattern of population 
means as specified by H; is true. If the hypothetical pattern is ten- 
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able, still another test will be needed of the further hypothesis (H;») 
that the vertical position of that pattern as specified by H; is true also. 


Procedure (a): The desired tests may be based on the two com- 
ponents of the sum of squares for “departure from hypothesis” 
[>7,;(M; — Myi)?], one of which is due to the “departure from pat- 
tern,” the other to the “vertical displacement” of that pattern. The 
sum of squares for “vertical displacement” is computed by 


[S2:(Mi — Mui) )?/N , 


while that due to “departure from pattern” may be obtained as a resi- 
dual from the “departure from hypothesis” sum of squares. 

The test of H;, (or the test for “departure from pattern”) may 
then be based on 


1 
2M — Bi)? — [Sn (M; — Mii) ]? / 
| P ’ 
J / *w 
in which a is computed as before. The d. f. for this F are (k — 1) 
and (N — k). 
i ’ 
Procedure (b): If the preceding test reveals that the hypotheti- 
cal pattern is untenable, there is of course no point in going further. 
However, if H;. is shown to be tenable, H, might still be false with 


reference to the vertical position of the pattern. The test for Hs, (or 
the test for ‘‘vertical displacement’’) may then be based on 





ges “ [0 (M, — Mi) ]*/0%s, 


for which the d. f, are 1 and (N — k). This F-test is the equivalent 
of a simple t-test of the significance of the deviation of the general 
mean (M = >n,M;/N) from the hypothetical M, = SniMui/N . 

It should be emphasized that the test for H;, should never be made 
unless H;,. has first been proved tenable. If either H;, or H;, proves 
untenable, H,; is untenable also. The tests for H;, and H;, together 
clearly constitute a better test of H; than does the over-all test (for 
departure from hypothesis ) alone. 


Case 6: Goodness of Fit of Curves Not Derived from 
Experimental Data (Single Sample) 


In an experiment designed like that in Case 2, it may be desired 
to test the hypothesis (H,) that the population means have certain 
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specified values, My, ,---, Mui,---, and My. , respectively. As in Case 
5, it is necessary to apply two separate tests, one of the hypothesis 
(H,.) that the pattern of the hypothetical means is correct, the other 
of the hypothesis (H,,) that the vertical placement of that pattern 
is correct. (Since these tests together will constitute an adequate 
test of H,, there is no need for an over-all test like that for H;, al- 
though a similar test could readily be supplied.) 


Procedure (a): In the & X n table referred to in Case 2, compute 
the “intervals < subjects” interaction variance (oF . ). For the same 


table, compute also 
n 
n> (M; RT [S(M; — M,y;)]? 





tj} 


which corresponds to the “departure from pattern” variance in Case 5. 
The hypothesis (H,..) may now be tested by 


n 
n> (M; — Mui)? =e (S (Mi — Mui) 1? 
F — Os ’ 
k—1 
for which the d. f. are (k — 1) and (n—1)(k—1). 
The assumptions involved are that the individual regression lines 
are all parallel and that deviations from individual regression are nor- 
mally distributed and of equal variance for all subjects. 


Procedure (b): Again consider the Y measures as arranged in a 
table of k columns (X-intervals) and n rows (subjects) as in Case 2. 
Compute the “between subjects” variance (0? ) for this table. 





If H.,. has been proved tenable, H,, may then be tested by 


ane 
, [> (Mi — Mu) J? 
P =x ; 
o*s 
for which the d. f. are 1 and (n — 1). 

The test for H,, may be regarded as a test of the signifi- 
cance of (M — M,,), in which M = 3M;/k is the general mean, and 
M, = >M,zi/k. The unit of sampling, so far as M is concerned, is 
the individual subject, rather than the individual observation. Hence 
the appropriate error term in this case is ov rather than i _ 


The assumptions involved are the same as in testing H,, . 
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Case 7: Goodness of Fit of Curves Fitted to the Experimental 
Means (Independent Samples) 


In an experiment designed like that in Case 1, it may be desired 
to test an hypothesis (H,) represented by a curved line which has 
been fitted to the experimental means. The procedure in this case 
would be exactly like that in testing H,;. (Case 5), except that the 
d. f. for the numerator of F would be k less the number of constants 
in the regression formula that were derived from the experimental 
means. For example, if Y = a log X + b had been fitted* to the 
means, the d. f. for the numerator of F would be (k — 2); if Y= 
aX* + bX? + eX + d had been fitted, the d. f. would be (k — 4), etc. 

In the case of a fitted regression line there would of course be no 
question of vertical displacement and hence no need for a test like 
that for H;,. 


Case 8: Goodness of Fit of Curves Fitted to the 
Experimental Means (Single Sample) 


In an experiment like that in Case 6, to test an hypothesis (H;) 
represented by a curved regression line which has been fitted to the 
experimental means, one would proceed just as in testing H.. , except 
that the d. f. for the numerator of F would be k less the number of 
constants in the regression formula derived from the experimental 
means. 


Case 9: Significance of Trend Differences: Successive Means Based 
on Independent Samples for Each Trend Separately 


The experiment in this case may be considered as consisting in 
effect of two’parallel experiments of the type in Case 1, k samples 
being selected from each of two populations (A and B). The results 
might be as represented in Figure 3, the solid line joining the ob- 
served Y-means for the samples from population A , the broken line 
joining the means of the samples from population B. 

The hypothesis (H,) to be tested is that the two population means 
coincide for each value of X. In other words, it is desired to know 


*For methods of fitting curved regression lines, see Fisher, R. A. Statistical 
methods for research workers, Edinburgh and London: Oliver and Boyd, 1938, 
Secs. 27-29, pp. 148-177 and Snedecor, George W. Statistical methods, Ames, Iowa: 
Iowa State College Press, 1940, Chapter 14 (Curvilinear regression), pp. 308- 
335. The methods there described are for fitting curved regression lines to indi- 
vidual observations, but of course are applicable to means as well. Being con- 
cerned with individual observations, however, these discussions do not suggest any 
tests based upon within-groups variance, such as are necessary if the tests are 
to be regarded as tests of adequacy of fit. 
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Criterion Measure (Y) 












Experimental Variable (X) 
Figure 3 


if the differences in the paired sample means can reasonably be joint- 
ly attributed entirely to random sampling fluctuations. 

The problem would of course be the same if two sets of k sam- 
ples were originally selected from the same population, and A and B 
represented different treatments, or different conditions under which 
the same treatment is administered. 

As in Case 5, two separate tests are required. 


Procedure (a): Consider the data as arranged in a table of 2 
(treatments) columns and k (X-intervals) rows. Compute the “be- 
tween treatments” (oF), the “treatments X intervals interaction” 


(07), and the “within cells” (0?) variances for this table. Assume 


that the Y measures are normally distributed and have the same vari- 
ance for all values of X and for both treatments. 


One aspect of H, may first be tested by 
F _ oy! % ¢ 


for which the degrees of freedom are (k — 1) and (N — 2k), N being 
the total number of Y measures for both sets of samples (N = Dia 
+ Sniz). This F tests the hypothesis (H,,) that the difference in the 
means of populations A and B is the same for ali values of X. (It 
does not test the hypothesis that these differences are zero, but only 
that the successive means of population A form the same pattern as 
those of population B. According to this hypothesis, the two lines of 
population means would either coincide or lie “parallel” to one an- 
other.) 
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Procedure (b): If Ho, proves tenable, another aspect of H, may 
be tested by 


= ¢g* /g? 
F=o7/s*, 


for which the degrees of freedom are 1 and (N — 2k). This tests the 
hypothesis (H,,) that the mean of the successive differences in popu- 
lation means is equal to zero. However, this mean could equal zero 
even though H,, were false; hence the test suggested in (a) preced- 
ing should be applied first. If either of these F’s is significant, the 
hypothesis (H,) is untenable. 


Case 10: Significance of Trend Differences: Successive Mean 
Differences Based on Independent Samples 


The experiment in this case is designed as in Case 1, except that 
for each value of X two criterion measures are secured for each sub- 
ject, one under Treatment A and one under Treatment B. A differ- 
ent and independent sample, however, is used for each value of X. 
The results may be represented as in Figure 3. The hypothesis (H,.) 
to be tested is that the population values of the treatment means are 
identical for each value of X. The two aspects of this hypothesis are 
tested separately. 


Procedure (a): The first hypothesis (H,..) to be tested is that 
the difference in population values of the A and B means is the same 
(not necessarily zero) for all values of X. 


For each value of X separately, consider the data as arranged 
in a table of 2 (treatments) columns and n; (subjects) rows. For 
each such table compute the “treatments X< subjects” interaction sum 
of squares (s8;xs,). Pool these sums of squares for all values of X 
and divide by the sum of the corresponding degrees of freedom. The 
result will be the “error” variance (oc? needed in this test. 


Bo 83 
Pcie, 
error N ‘abhi k 


in which N is the total number of subjects (VN = 7n;), not the total 
number of observations, which is 2N . Now consider the data for the 
entire experiment as arranged in a table of 2 (treatments) columns 
and k (X-intervals) rows. For this table, compute'‘the “treatments < 
intervals” interaction variance (07) and the “treatments” variance 
(02). 

The hypothesis (H,.,.) may now be tested by 


Si 
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F=¢e /s* ; 


TxI error 
with (k — 1) and (N— k) d.f. 


Procedure (b): On the assumption that H,.. is true, one may 
next test the further hypothesis (H,.,) that the mean of the succes- 
sive differences in population means is zero. (This is equivalent to 
testing the significance of the difference in the general means for 
“treatments.”’) : 


This hypothesis is tested by 
F= eh o F 


error 


with 1 and (N—k) d.f. 
If either H,o. or Hi.» is untenable, H,. is untenable also. 


Case 11: Significance of Trend Differences: Successive Means 
Based on Single Sample for Each Trend Separately 


An important type of experiment is that which consists, in effect, 
of two parallel experiments of the type illustrated in Case 2. That is, 
k successive measures of the criterion are obtained for each individ- 
ual in each of two independent samples, each sample being drawn 
from a different population (or each given a different treatment, or 
given the same treatment under different conditions). Figure 3 could 
again represent the experimental results, with the difference that in 
this case all of the A means are based on the same individuals( in- 
stead ‘of on k independent samples), and that all the B means are 
similarly based on a single sample (independent of the A sample). 

The hypothesis to be tested (H,,) is that the population means 
coincide for each value of X. 

Two separate tests, corresponding in purpose to those described 
in Cases 9 and 10, should be applied, as follows: 


Procedure (a): The first step is to test the more specific hypo- 
thesis (H,,,) that the difference in the A and B population means is 
the same (not necessarily zero) for all values of X. 


For this purpose, first regard all of the Y measures as arranged 
in a table of 2 (treatments) columns and k (X-intervals) rows. Then 
compute the “treatments X intervals interaction” variance (7 ) for 


this table. 

Now consider the A-data alone as arranged in a table of k rows 
and », subjects, and compute the “intervals < subjects” sum of 
squares (88;,s:4)) for this table. In the same manner, compute (88;.s;3) ) 
for the B-data alone. Then pool these sums of squares and divide by 
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the sum of the corresponding d. f. to secure the “error” variance 


(oe 


error 







, a ie + 88ixs:B) 


error (m4 + Ng —2) (k—1) F 













This involves the important assumption that the individual re- 
gression lines are parallel for all members of both populations and 
that the deviations from individual regressions are normally distrib- 
uted and of equal variance for all members of both populations. 


The hypothesis (H,,,.) may then be © tested by 
2 /o 2 


TxI error’ 











for which the degrees of freedom are (kK — 1) and (m4 + mz — 2) 
(k—1). 


Procedure (b): If Hi. is proved tenable, the next step is to test 
the hypothesis (H,,,) that the mean of the differences in population 
means for the various values of X is zero. It should be apparent 
that this is equivalent to testing the significance of the difference 
(M, — Mz) in the general means of the two samples. 


















For this purpose, consider the data for the A-sample above as 
arranged in a k X n, table, and compute the “sum of squares for be- 
tween subjects” (s8s,4)) for this table. Treat the B-data similarly to 
compute s8g,z) . 

If n, and nz are small, the significance of (M4 — Msg) may be 
tested by 


88g¢4) + S8g:z) 1 1 
t= (M,—M - (+=), 
( . 2) os + Np ima 2) Na Np 
for which the number of degrees of freedom is (m4 + mg — 2). 


If n, and nz are large, the significance of (M4, — Mz) may be 
tested by the normal deviate (critical ratio) 


S8g(4) 8Ss(B) 
R.=(M,—M i 
en dae ats k (ms — 1) 


























avoiding the necessity of any assumption of equal “between subjects” 
variances for the two populations. 

If either H,,, or H,,, must be rejected, then, of course, H,, must 
be rejected also. 
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Case 12: Significance of Trend Differences: Successive Means 
Differences Based on a Single Sample 


The design of the experiment in this case is like that of Case 11, 
except that both treatments are administered to the same sample of 
subjects, instead of one to each of two independent samples. Again 
the purpose is to test the hypothesis (H,.) that the population means 
for the two treatments are identical for all values of X or that there 
is no difference in the two treatments. Figure 3 could again repre- 
sent the experimental] results—all means, for both A and B,, in this 
case being based on the same individuals. 


Procedure (a): In terms of the methods of analysis of variance, 
this is a factorial design involving the three factors: “treatments,” 
“intervals,” and “subjects.” As in the preceding cases, two separate 
tests are needed. 


The first step is to test the hypothesis (H,.,) that the differences 
in A and B means for the various values of X all have the same popu- 
lation values. 

Using the methods of analysis of variance for factorial designs, 
compute the “treatments” variance (c*), the interaction variance 


for “treatments X intervals” (0° is the interaction variance for 


“treatments < subjects” ate ), and the higher order interaction for 
“treatments < intervals < subjects” (0° E >», 


x8 


The hypothesis may now be tested by 
f= a /s, : 


TxIx8S 
for which the degrees of freedom are (k — 1) and (k—1)(n—1). 
Procedure (b): The next step is to test the hypothesis (H,.,) 


that the mean of the differences in A and B means is zero for the en- 
tire population. 


This hypothesis would not be tested unless H,., had first been 
shown to be tenable. On the assumption that there is no real T X J 
interaction, the hypothesis H,., may be tested by 


r= o* /o . 


Tx8 


for which the degrees of freedom are 1 and (n — 1). 
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THE ESTIMATION OF RELIABILITY WHEN SEVERAL 
TRIALS ARE AVAILABLE* 


HOWARD W. ALEXANDER 
ADRIAN COLLEGE 


In certain types of experiment it frequently happens that we 
have a block of data consisting of several trials on the same individ- 
uals. Using the methods of estimation provided by the analysis of 
variance, estimates of reliability are derived for this case, and the 
conditions under which each is valid are discussed. Various relations 
between these estimates and the product-moment coefficient of corre- 
lation are obtained. 


It frequently happens in educational, psychological or biological 
research that it is possible to obtain repeated measurements on a 
limited number of subjects over a considerable period of time. Sup- 
pose, for example, that we have 12 experimental subjects maintained 
under constant “standard” conditions, and that we obtain 6 trials of 
a certain measure (say an intellective test) spaced two days apart. 


From the block of data thus provided, consisting of the scores of 12 
individuals on six trials, can we estimate the reliability of the test? 
How will such an estimate compare with other measures of reliability 
obtained from just two trials? 

In many types of biological experiment it is difficult to obtain 
large numbers of subjects. Thus estimates of reliability based on the 
accepted methods may suffer the serious disadvantage of being based 
on small “degrees of freedom.” It is of considerable importance to 
know whether, and to what extent, this handicap can be compensated 
for by a larger number of trials. The present paper will discuss meth- 
ods of estimating reliability when a relatively large number of trials 
is available on a relatively small group of subjects. 

In Part I (sections 1 to 8) methods of reliability estimation based 
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on the analysis of variance are developed. In this development, the 
case of two trials is used in order that the estimates derived may be 
compared with the more usual measures of reliability. Emphasis is 
placed on the use of estimates of population variance as the funda- 
mental entities from which estimates of reliability may be constructed. 

Part II (sections 9 to 13) deals with the case of more than two 
trials. A genera] test for trend is outlined. Equations for the esti- 
mation of variance when a component of linear trend is present are 
developed. Finally, the estimates of reliability are studied from the 
standpoint of average correlation and in the light of maximum like- 


lihood. 


Part I 


1. The concept of reliability 

The concept of reliability has been very adequately discussed in 
a publication by R. W. B. Jackson (5), in which a rather complete 
bibliography is provided. The reliability of a test or measure is con- 
cerned with its accuracy. Thus, some estimate of the error of meas- 
urement of the test will be included as one component of any measure 
of reliability. “Error of measurement” may be taken to mean, for 
example, the failure of the individual to repeat his performance un- 
der conditions as nearly identical as possible. For a single individ- 
ual, this may be measured by the variance of his scores; for a group 
we may use the average variance for all individuals. Under other cir- 
cumstances, the error may be measured by the deviation of the indi- 
vidual from the average performance of the group, in which case it 
is essentially a measure of the inconsistency of the performance of 
the group. In this case, the appropriate quantitative measure of error 
is readily obtained by the methods of analysis of variance from the 
quantity known as the “interaction mean square.” 

The measure of error may be reported in absolute units. Or it 
may be expressed as a percentage of the mean of the sample from 
which it was derived, a procedure common in the physical and bio- 
logical sciences. Or, again, it may be compared with the inter-indi- 
vidual variability; this alternative gives rise to the product-moment 
coefficient of correlation, to the intraclass correlation, and to certain 
other measures. 


2. The basic data for reliability estimation 

Consider an aggregation of measurements which have been re- 
peated & times on n individuals. These measurements may be ar- 
rayed as in Table 1. 
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TABLE 1 
Trials 


8 
a 4 
x. 
x 


n X,, 
If the subscript 7 = 1,2,3,---, m designates the individual, and the 
subscript h = 1,2,3,---, & designates the trial or repetition, then 
the score for any individual on any trial may be designated by X ja. 

In some cases the repetitions or trials are obtained over time; in 
other cases (as in certain kinds of psychological tests) duplicate meas- 
urements are obtained by splitting the odd and even items of a test 
into two groups. The term “individual” can be applied to objects as 
well as persons. For example, in a geological investigation we might 
consider the “individuals” to be different kinds of gravel; in this in- 
stance we might use a large number of samples of the same kind of 
gravel, rather than repeated measurements on the same sample. In 
many kinds of biological experiment, however, it is more convenient 
to repeat the measurements frequently on the same individual. 

As will be brought out in later sections, the choice of a suitable 
measure of reliability is dependent upon the presence or absence of 
trend. We may define trend, in a table of individuals by trials such 
as Table 1, to be a fluctuation of the sample mean from trial to trial 
that is too large to be attributed to the error of measurement. Jack- 
son (4) has used the term “trial effect” or “practice effect” for what 
we have named trend, and uses methods of analysis of variance to 
test for it. 


3. The test for trend in the case of two trials 


Throughout the remainder of Part I we shall be concerned pri- 
marily with the case of two trials, which may be displayed as in 
Table 2. 


TABLE 2 
Trials Difference 
2 
X19 
Xo d, 
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We have supplied a difference column, d; = X;, — Xj.. In the case of 
two trials, a simple test for trend is provided by the t-test for paired 
variates, which is a test of the hypothesis that the mean of the first 
trial is the same as the mean of tfie second trial. Numerically, it is 
given by the formula 
_ Vn—1 (S24) 
Vn Sd — (Say? 

The value of ¢ is to be compared with tabulated values for n — 1 df 
(degrees of freedom). However, it is preferable to use the equivalent 
F-test, since in preparing to calculate F we find other quantities which 
are needed in the estimation of reliability. 

The calculations required to set up the table of analysis of vari- 
ance are described in numerous texts (8, Chap. 11). The features in 
which we are most interested are illustrated in the two fictitious ex- 
amples in Table 3. 





(1) 





TABLE 3 
Example ] Example 2 
Y X+Y X—Y X+Y X—Y 
67 121 








92 








1318 


Note that the tables have been constructed in such a way that 
the column of X-values is the same in both examples, while the Y-val- 
ues in Example 2 are 10 less than the corresponding Y-values in Ex- 
ample 1. The two analyses are as given in Table 4. 


TABLE 4 
Example 1 Example 2 
Source of variation df Mean square df Mean square 
Between individuals 11 ; —1,476.076 11 ; ——1,476.076 


Vv 
Between trials “ie Ye = 167 1 + = 620.167 
Interaction 11 Vi,— 51.348 11 it 51.848 





it 








yf 


t 
is 
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Only the between-trials mean square has been affected by the ad- 
justment of the Y-values in Example 2. The F-test for the presence 
of trend is, for the case of two trials, equivalent to the t-test already 
given, and is provided by the ratio F; = V;/V;;. For the illustrative 
examples we find: 


Example 1: F;=.003 Example 2: F; = 12.08 


Reference to Snedecor’s table of F (7) shows the first of these two 
values is non-significant while the second is significant (that is, the 
value of F; = 12.08 lies beyond the 1 per cent value of F, for 1 and 
11 df). We conclude that there is a significant trend in the second 
case, but not in the first. The trend has been introduced by the uni- 
form displacement of the Y-values in the second example. 


4. Components of variation and estimates of variance 

We now introduce a set of assumptions basic to the development 
of the various measures of reliability. Returning to the case of k 
trials, we postulate that each of the observed values in a table such as 
Table 1 can be expressed as the sum of four parts: 


Xj,—M +1; + T + Ry, 
where 


M =the population mean, constant for all individuals on all 
trials, 
I; = a component characteristic of the particular individual, but 
constant over trials. 
T,= a component characteristic of the particular trial, but con- 
stant over individuals, 
R,;, = the residual, which does not vary systematically either with 
individual or with trial. 


We assume that the components J;, 7, and Rj, are normally dis- 
tributed and uncorrelated in the parent population, with mean zero 
and variances o;?, o;?, and o,?, respectively. These three population 
variances will be referred to as “between individuals variance,” “be- 
tween trials variance,” and “error variance,” respectively. For the 
validity of the methods which will be developed, it is necessary to 
assume, further, that the individuals constitute a random sample from 
the population of such individuals and that the trials constitute a 
random sample from the population of all similar trials. A more com- 
plete discussion of several of these assumptions is given in (2), which 
also contains a useful bibliography. 

It is natural to ask whether a consecutive series of trials can ever 
be independent, or whether a given set of trials can ever be regarded 
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as a random sample from the population of trials. An experimental 
example will be described in some detail to illustrate these possibil- 
ities. Twelve normal young men were maintained on a diet as con- 
stant in its principal components as was consistent with reasonable 
variety. On six occasions several days apart electrocardiagrams were 
obtained before and after the noon meal to determine the effect of the 
meal] on the various electrocardiagraphic measures. Since all the ex- 
ternal factors were kept as constant as possible during the experi- 
mental period, any trend that might appear would properly be de- 
scribed as random, in the sense that it was uncorrelated with any con- 
trollable external factor. A statistical test for trend was applied to 
one of the electrocardiagraphic functions known as “sigma T”’, and it 
was found that while there was no linear or cumulative trend, there 
was a significant trend common to the group, presumably attributable 
to the differences between the six meals. The detailed analysis is pre- 
sented in (1). It is quite clear that the meals were chosen randomly, 
since the meal sequence was planned to be as homogeneous as possible, 
and there was no attention paid to the possible effect of any dietary 
components on electrocardiagraphic performance. Such random in- 
fluence of significant factors can doubtless occur in many situations. 

‘ While the population variances cannot be calculated from the in- 
formation in the sample, unbiased estimates s;?, s,2, and s,? of o;7, o:7, 
and o.”, respectively, can be obtained from the equations 


V; = 8,? + k 8;? 
V, =8 + Nn 8;? (2) 
Vit — 8.7 


(see, for example, 7, section 10.6). 

The F-test, F; = V:/Vi:, may be used to test for trend in this 
case as in the simple case of two trials. Note that 

8." + N 8,” $;? 
ee. ae +a( ) ’ 

' which shows that when F; = 1, s,2 = 0. This makes plausible the 
fact that F, can also be regarded as a test of whether s,? is signifi- 
cantly different from zero. If s,? is not significantly different from 
zero, the component 7), is unnecessary, since there is no trial compo- 
nent which can be distinguished from error variation. Under these 
circumstances we replace V; and V;; , which are both estimates of er- 
ror, by the pooled estimate known as the “within individuals” mean 
square, V,,;: 


1 
Vos =— (Vi + (n—1) Vin), 
n 





Sn a a a 
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with n(k — 1) df. For Example 1 we obtain the analysis: 


Source of variation df Mean square 
Between individuals 11 Vi =1,476.076 
Within individuals 12 Vui= 47.083 


Since the hypothesis that o,? = 0 has been confirmed by the test, F; , 
we drop it from the equations of estimation, and equations (2) are 
replaced by the following pair of estimation equations: 


Vi =s'2 + ks’? 





3 
Ver =s'2, (3) 
where s’;? and s’.? are new estimates of o;? and o.”. In the case of Ex- 
ample 1, equations (3) yield s’;? = 714.496, s’.? = 47.083. 


5. Estimates of reliability 

All the estimates of reliability that will be developed are ulti- 
mately derived from the product-moment coefficient of correlation. 
Jackson has shown (5, Appendix A, pp. 107-112) that under circum- 
stances which arise frequently estimates of reliability other than the 
product-moment coefficient are preferable (see sections 7 and 12 be- 
low). In the present section we shall develop the purely algebraic 


properties of four estimates of reliability, the uses of which will be 
discussed later. Of these, all except the first are given for the gen- 
eral case of k trials. 


The product-moment coefficient of correlation 


(a) Consider Table 2, with columns of X;,, Xj., and d;. Let 
$,? , 87, and s,? be the unbiased estimates of the variances of these 
three columns: 


ates tas (Xj)? ’ PRS) ae (SX je)? 
n(n—1) n(n—1) 

__ n Xd? — (Sd;)? 

a n(n—1) ; 

It may be shown, by simple algebra, that the following equations hold 

for the case of two trials: 


Vi,=s8,? + 8*9—3387, Vi =—s?—} 87, 











877 


(4) 


8,27 = $(8,? + 8.2 — 3,7). 


Finally, the well-known formulas for the product-moment correlation 
lead to 
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s,* ah. g.* ee 87 8;? 


(5) 





T= — m 
26-2. 8,82 


Using these formulas we find, for both Example 1 and Example 2: 


8,° = 816.909, s.°=—710.515, 8s? = 102.697, 
8;? = 712.364, r=.9350. 


Note that the product-moment coefficient is insensitive to the shift in 
the values on the second trial. It does not reflect the presence or ab- 
sence of trend. 


Intraclass correlation: symmetrical table estimate. 

(b) R.A. Fisher, in his Statistical methods for research work- 
ers, (3, 207) makes use of two different forms of the intraclass corre- 
lation. The first is estimated from a symmetrical table, using the for- 
mula of Harris: 


kS (4, —#)?= (n—1) 8? (1+ (k—1)r"], 


where r” is the symmetrical table estimate of the intraclass correla- 
tion, % is the over-all mean, Z, is the mean of the scores for the p“ in- 
dividual, and s? is an estimate of the total variance. The value of r” 
may be expressed in terms of V; and V,;: 


see (n—1)Vi—2V iyi 


—~ Sas ’ 6 
(n—1) Vi t+n(k—1)V qi = 





or in terms of the ratio F’; = Vi/Vuwi: 


fF’; —n/(n—1) 
"SS erate (7) 
F’; + n(k—1)/("—1) 

It will be shown later that this estimate is really only appropriate for 
a case such as Example 1, where no trend is present. For Example 1, 
we find r” = .9327, which is a fairly close estimate of the product- 
moment correlation, 7 = .9350. If we calculate r” for Example 2, we 
find vr” = .8640, which is a serious underestimate of 7. The effect of 
the presence of trend is to depress the value of r”. 


Intraclass correlation: unbiased estimate. 

(c) The unbiased estimate of the intraclass correlation, which 
will be denoted by 7’, takes a very simple form when it is expressed in 
terms of the estimates s’;? and s’.? of equations (3): 


(8) 


»’ 2 2° 
e373, 
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This form is susceptible of an intuitive interpretation. The numera- 
tor is an estimate of the between-individuals variance, while the de- 
nominator is an estimate of the total variance. Thus 7’ measures the 
fraction of the total variance that is attributable to inter-individual 
variation. More convenient for computational purposes are the ex- 
pressions 


so Vi eae! V wi 
Vp ee Fs 





- 
9 
F’,—1 (9) 
= ———., where F’';=—Vi/Vwi. 
F’,+k-—1 
Again, this estimate is suitable only when there is no trend. However, 
we present its values for both Example 1 and Example 2: 


Example 1: r’ =.9382, Example 2: r’ =.8746. 


Note that, as with the previous estimate, 7’ is rather close to r when 
there is no trend but underestimates 7 when trend is present. 


Intraclass correlation adjusted for trend. 
(d) A fourth form of estimate will be useful. It occurs, for 
the case of two trials, in Jackson’s monograph on reliability (6, 110, 
formula 83). We shall denote by 7* and describe it as the intraclass 
correlation adjusted for trend. We may define it initially in terms of 
the variance estimates of equations (2): 
Vig 
_— : (10) 
8? + 8? 
Although this has the same form as equation (8), it must be clearly 
distinguished from the equation for 7’. The primed estimates of equa- 
tion (8) are derived from equations (3), for the case without trend. 
The estimates of equation (10) are the same as those in equations (2), 
and are for the case with trend. In this case the denominator is not 
an estimate of the total variance, since the trend component s,’? does 
not appear; it is the “total variance apart from trend.” 


Some justification for the name given to this estimate may be 
offered. Suppose that an experiment were designed for the purpose 
of estimating reliability, using a number of trials. It was intended 
that performance should be kept steady, so that no trend would be 
present and the intraclass correlation could be used. Suppose it were 
found that a certain amount of trial fluctuation had entered. Then 7* 
may be regarded as an estimate of the value that would have been 
obtained if the fluctuation had been avoided. 








ee 
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Equations (10) and (2) lead to simpler computational forms: 





. Vi—Vie 
T= 
Vit (kK—1)Vie 
11 
F.—-1 (11) 


= ————_., where F,=Vi/Vit. 
F,+k-—1 


Note that the above expressions are closely analogous to those in (9). 
It may be noted here that F’; = V:/Vi is a test of whether 7’ is 
significantly different from zero, while F; = V;/Vi; is a test of wheth- 
er 7* is significantly different from zero. 
For both Example 1 and Example 2 we find 


F,=28.75, 1°=.9328. 
Thus 7* in this case provides a fair estimate of r = .9350 and is in- 
dependent of the presence of trend. 


6. Algebraic relations between the estimates 

(a) Relation between r and r*. 

For the case of two trials, r 2 r*. For proof, note that from 
equations (4) it follows that (for k = 2) 

V; — Viz =2 8;7, V; + Viz = 3,7 + 847. 
Thus, for k = 2, we find 
" Vi-— Vie 2 8;? 
or” Se ma 
V; + Vit s.* + 8,* 





(12) 
8;? 


~ (8,2 + 8,4) 


The expression in the denominator is the arithmetic mean of the vari- 
ances of the two trials. On the other hand, in the expression 


= 8;?/8,8, 


of equation (5), the denominator is the geometric mean of the vari- 
ance of the two trials, while the numerator is the same as in (12). 
But the identity 

4(3,? + 8,7) = 8,8, + $(8, — 8,)? 


shows that the arithmetic mean of the variance is somewhat greater 
than the geometric mean, except when the variances are equal, in 
which case the arithmetic and geometric means are also equal. From 
this it follows that r 2 7”. 
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(b) Relation between r* and r’. 


We shall show that, for all values of k , 
r>r if F;>1, 
r=r if F,;=1, 
r<r if F;<.1. 

We first note the identity 


F;—1 
F,=F',(1 + : 





F’; 
pear, 8) 
n n 


which is a consequence of the relations F; = V;/Vi; , F’; = Vi/Vui and 
1 

Vwi = — (Vi + (n—1) Vix). 
n 


Using the above identity, we may write 
F’; 
cia: Ht ome Oi, eZ 
: F; eee 1 i n ( t ) 


== = 
F;+k—1 








F’; : 
F’;+k—1+— (F;—1) 
n 


Comparing this with the expression (9) for 7’, we see that r* is ob- 
tained from 7’ by adding (if F; > 1) the same positive quantity to 
numerator and denominator of the fraction, which has the effect of 
increasing its value. Thus, if F; > 1,%° > 7’. The other relations are 
proved similarly. 


(c) Relation between r’ and r’. 
For all values of k, r’ > r” . To see this, we note that the expres- 
sion (7) for r” may be written 











1 
F,-—1—-— 
ss n—1 
= 4 
k—1 
F,+k-—1+ 
n—1 


Comparing this with (9), we see that r” is obtained from 7” by sub- 
tracting a quantity from the numerator and adding a quantity to the 
denominator, which has the effect of reducing the value of the frac- 
tion. Thus 7’ > r”. 


7. Maximum likelihood estimates 

R. W. B. Jackson has shown (6, 107-112) that, for the case of 
two trials, the maximum likelihood estimate (MLE) of the correla- 
tion between two trials depends upon the presence or absence of trend 
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and the equality or inequality of the population values of the vari- 
ances of the two trials. He distinguishes three cases: 


(a) Trend is present and the variances of the two trials are un- 
equal. In this case, Jackson shows that the MLE is given by 
the product-moment coefficient of correlation. 

(b) Trend is present and the variances are equal. Here the MLE 
is shown to be given by an expression algebraically equiva- 
lent to 7* as defined in (11). 

(c) There is no trend and the variances are equal. The MLE is 
found to be given by the expression (6) for r”. 


The last two of these results will be generalized to the case of k trials 
in section 12. 


8. A geometrical note 


For the case of two trials, an interesting geometrical comparison 
may be made of the various estimates. Let the two trials be repre- 
sented by two parallel vertical lines, and let the scores be graphed 
upon these lines. Then if we join corresponding points on the two 
lines, we get a series of n lines, one for each individual. Parentheti- 
cally, it may be noted that if Vi, — 0, the m lines will be parallel. 
What conditions are imposed upon the lines if the different estimates 
of reliability are put equal to unity? The three parts of Figure 1 il- 
lustrate the situation. 

FIGURE 1 
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fart wl f=" =1 
The condition r = 1 requires that the n lines be convergent (which 
includes parallelism as a special case), r* = 1 requires that they be 
parallel, while r’ = r” = 1 requires that they be horizontal. Thus the 
most stringent condition is 7’ = r” = 1, and the least stringent is 
r =1. Morton Jellinek (7) points out the fact that 7’ takes its maxi- 
mum value only when the values on both trials are identical, offering 
this as a reason for the adoption of r’ as a measure of reliability. How- 
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ever, Jackson’s result makes it clear that the use of 7’ is justified only 
when no trend is present. The developments below will show that this 
result holds also for the case of k trials. 


Part II 
9. A general test for trend 

We must now define more specifically what is to be understood 
by the term “trend” when we are dealing with more than two trials. 
The test for trend will be dependent upon the assumptions which are 
made regarding the distribution of observed values. 

Consider, first, the test for trend which results when the basic 
assumptions are those of section 4. The ratio F; = V:/Vi+ provides, 
in this case, a test of whether the trial means are different. This is a 
suitable test for trend under the assumptions above, but some atten- 
tion must be paid to the restrictive effect of these assumptions. 

Consider the implications of the assumption that 7; is a normally 
distributed random variable with mean zero. This implies that on any 
particular trial the mean is just as likely to fall below the population 
mean as above it. Thus, under this assumption there can be no cumu- 
lative trend. These considerations make it clear that the set of as- 
sumptions of section 4 are not fulfilled in a situation involving learn- 
ing, for if later scores tend to be higher, as with learning, the means 
cannot be randomly‘ distributed. 

Another consequence requires emphasis. Suppose that the experi- 
ment were repeated a great many times with the same individuals, 
who will be supposed not to change between repetitions. For each in- 
dividual on each trial there would thus arise a population of values 


distributed about a certain mean value X;,. It is a consequence of our 


assumptions that if we graph the & trial values X;, for each individual - 
against time and join these points by a broken line, then the n graphs 
for the individuals of the group will be parallel, that is, any one of 
them may be made to coincide with any other one by vertical displace- 
ment. 

If we have a priori reason for believing that the individual graphs 
will not be parallel, then the assumptions of section 4 are unsuitable. 
This constitutes an additional reason for regarding these assumptions 
as inappropriate for a situation in which learning is present, since 
the likelihood is that individuals will learn at different rates. How- 
ever, the experimental example described in section 4 indicates that 
examples may be encountered in which we have no reason for distrust- 
ing the assumptions of section 4. 

It is important to be able to test for cumulative as well as for 
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random trend. There are many types of learning function that could 
presumably be employed. In most circumstances a linear learning 
function is probably adequate, and the discussion in the present paper 
will be confined to the case of linear trend. We shall modify the as- 
sumptions of section 4 by postulating that a linear term is present in 
addition to the components already described. Thus the observed val- 
ues may be partitioned into five components: 


Xp—M+1; + T, + GB; + Rp, 


where M , 1; , T, and FR; are as previously defined, and 
B;=a slope term characteristic of the individual, 
t, = the time of the h* trial, measured in any conveni- 
ent units. 
For computational convenience, the origin of the time variable is 
chosen to coincide with the mean value of 4, so that St,=— 0. 

The analysis of this case is more fully discussed in (1), where 
tests for trend are developed for the general case and for the case in 
which the term 7’, is deleted. In this account only the general case 
will be considered. 

The computational details may best be indicated by starting with 
the simple analysis of variance such as was used in section 3. We use 
the notation SS; , SS; , and SS;; to denote the sums of squares corre- 
sponding to V;, V;, and Vi,. Thus we have the table: 





Source cf variation Sum of squares df Mean square 
Between individuals SS; n—1 V; 
Between trials SS, k—1 V; 
Interaction SS;, (n—1)(k—1) Vit 


In addition to these quantities, we require to compute for each indi- 
vidual the sum of products P; = St;-Xj;, (summed over trials). We 
also compute K = St,? (summed over trials) and the quantities 


1 1 
=— (SP;)? and R=—SP;, 
Q aK (SP;) K Fi 
both summed over individuals. 


The analysis yields five mean squares, and the table of analysis 
of variance is as follows: 





Source of variation Sum of squares df Mean square 
Error SS;,+Q—R (n—1)(k—2) Vv. 
Group deviations from 

linearity SS, — Q k—2 Voapt 
Group slope Q 1 Vos 
Between individual slopes R—Q n—1 Vis 


Between individual means SS; n—1 V; 





Re 


er 


il- 
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From this analysis there result two tests for trend: 


(a) Fart = Voan/Ve is a test of whether the whole group shares 
a common pattern of variation in addition to the linear component 
which may vary from individual to individual. 
(b) Fy, = Vz./V- is a test of whether there exists a significant 
group slope. 
As an example, consider the analysis of the work pulse rate for 
35 normal young men. Pulses were determined after five minutes on 
a motor-driven treadmill, on three occasions, with intervals of one 
week between the trials. The trial means were: 
Trial 1 Trial 2 Trial 3 
135.6 136.9 132.8 





The simple analysis of variance is 





Source of variation Sum of squares df Mean square 
Between individuals SS; = 16,928.63 34 V,; = 497.90 
Between trials SS, = 305.20 2 V, 152.60 
Interaction SS;,= 3,240.80 68 Viz 47.66 


The auxiliary quantities required for the complete analysis are 
Q=145.73, R=2424.50. 


Using these, we obtain the analysis 





Source of variation Sum of squares df Mean square 
Error 962.03 34 VV. = 28.295 
Group deviations from linearity 159.47 1 V oat = 159.47 
Group slope 145.73 1 Vog 145.78 
Between individual slopes 2,278.77 34 Vi, = 67.02 
Between individual means 16,928.63 34 V; = 497.90 


F jazi = Van /Ve= 5.64, F,—=V,/V,=5.15. 


Both of these F-values are significant, so that there is both a sig- 
nificant group slope and a significant variation about the group slope 
which is shared by the whole group. The existence of significant 
group slope means that the simple analysis of variance is not valid, 
since the trial means are not randomly distributed about the grand 
mean. 


10. Estimation when trend is present 
When the tests of significance of section 9 show that linear trend 
is present, then neither of the estimation equations (2) or (3) is 
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valid. Let us suppose that tests of significance have shown that each 
term of the equation 


Xp —M+1,+ 7, + eB; + Ri; (13) 


represents a significant type of variation. Let s;?, s;7, s,?, and s,? desig- 
nate unbiased estimates of the variances of the terms I; , T;,, B; , and 
R,,. The mean values of I; , T;, , and R;, are assumed to be zero. Then 
we obtain the estimation equations 


Ve =8,? 

V gapt =—=s7t+n S;? 

Von 8,2 +n82 + K 8,? + nKb? (14) 
Vis = 8," +K Sp? 

Vv; =e +ke,?, 


where K is as defined in section 9 and b is the mean value of B;. 

If tests show that the random trial term T;, may be omitted, the 
analysis must be altered to the extent of pooling V. and Vgay: to ob- 
tain a new term which may be called the “deviations from regression” 
mean square, given by 


SS; + SSi: +Q—R 





Va,= 
“ n(k — 2) 
The estimation equations must also be revised: 
Varr —_ " fg 
Vee — 82+ K 8',? + nKb”? 
Vis = 82 + K 8,7 (15) 
V; => a’? + k gs’ ;? ’ 


where s’,”, s‘,”, s';2, and b’ are new unbiased estimates. Note that the 
relation between equations (15) and equations (14) is similar to that 
between equations (3) and equations (2). 

It should be observed that the equation for V; has the same form 
in each of the four sets of estimation equations (2), (3), (14), and 
(15), namely, V; = s.2 + k 8;?. In each case, s,? is the measure of resi- 
dual or unallocated variation, after the variation due to all assignable 
sources has been eliminated. From a slightly different standpoint, it 
is a measure of the failure of the performance to follow some law 
specified by an equation such as 


Xj, — M + I; + T;, + tyB; 9 
or some simpler law. There is some justification for accepting s,? in 


the more complicated cases represented by equations (2), (14), and 
(15) as an estimate of the error term V,,; for the simple case in which 
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there is no trend. By the same token, the intraclass correlation will, 
in all cases, be estimated by 


ia v—), Pe -1 
= ee . (16) 


s?+s2 V;+(k—1)V. F,+k—1 





where F'; = Vi/V. and V, is the measure of residual variation. 

It would, of course, be possible to extend equation (13) by in- 
cluding terms of higher degree in t, , such as 4, t,, etc. This would, 
from the standpoint of reiiability, merely have the effect of refining 
the error term, V., removing from it all variation attributable to 
sources which act through all the individuals of the group. It is to 
be expected that the reliability would still be estimated by equation 
(16) ; the algebraic verification of this possibility is rather compli- 
cated, however, and has not yet been carried out. 

The use of formula (16) may be illustrated by means of the ex- 


ample of section 9. If we use the simple analysis of variance we ob- 
tain F’'; = 497.90/47.66 = 10.45 and from formula (11), 


r* = 9.45/12.45 = .759 . 


On the other hand, using V. = 28.295, the error term from the analy- 


sis with the linear component, we have P,= 497.90/28.295 = 17.59, 
and from (16), 


r = 16.59/19.59 = .847. 


In the first case, the error term included a component of variance 
which could legitimately be separated from it, with the result that 
we had overestimated the error and thus underestimated the reliabil- 
ity. 


1l. r* as average correlation 

The close relation of the quantity 7*, the intraclass correlation 
adjusted for trend, with the product-moment coefficient of correlation 
is brought out by the following theorem: 
Theorem. If, in a table of n individuals by & trials the variance of 
each trial is the same, say s?, then the average of the k(k — 1)/2 
values of the product-moment correlation, calculated for every pair 


Vi-— Vit 
Vit (k—-1)Vit- 


Proof. Put d,, = X;, — Xj,, where p and q designate any two col- 
umns of the table. Then the estimated variance of d,, is 





of trials, is given by 7* = 
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1 
8pq? = ———— [n Dd,,? — (Sdyq) 7]. 
n(n—1) 
It will be convenient to employ the following notation: 


Column totals: U, =>Xin 
i 


Row totals: W;=DXin 
h 
Grand total: G = DX. 
ish 
Then we may write 
1 
8pq? = ————  [" 5 (Xjp — X5q)? — (Up — U,)?).. 


~ n(n—1) 
On the other hand, if 7,, designates the correlation between columns 
p and q, from equation (5) with s,? = s,? = s? we obtain 


28? — 85,7 Spq" 
Tq = =1—-— 
2s? 2s? 
Thus the average value of the correlation, 7 , is given by 
§? 
7=1-——, 
2s? 


where § is the average of the k(k — 1) /2 values of s,,? for all pairs 
of columns: 


2 
# =——__ Js, 
k(k—1) dq 


. 
= [nS (Xjp— Xjq)? — ES (Uy— U,)?]. 

we-vi—)'a  a 
In order to simplify this expression, we use the identities 


D(X jp — Xjq)? =KE Xp? —TW;’, 
jh j 





>(U,— U,)?=k> U;? — G?. 
h 


Since the proofs of these identities are quite analogous, we prove only 
the second: 


S (U,—U,)?= (U;— U2)? + (Us — Us)? + (Ui, — U)? ++ 


= (k—1) (U,? + U,? + Us? + --- + U;?) 
— 2U,U, — 2U,U; —2U,U,—.::-. 
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But 


G? = (U, + Uz, +++» + Ux)? 
= U,? Ss U.2'+ ee + U;2 — 2U,U.2 oo 2U,U; + 2U,U, + re S 


Hence 
> (U,— U,)? =k(U,? + U2 + --- + UZ) —@=k> U;? — G. 
pq h 


Using the two identities, 





z= 1 [ sxe Ewe—=z Ue + S| 
~ (n—Dk—-DL I” ORG on nk 


But the expression in the square parenthesis is simply the “interac- 
tion sum of squares,” which is given by (n — 1)(kK—1)Vi,. Thus 








2 
52 — -(n—1) (K—1) Vig = 2V. 
ey ee a 
and 
5 3 Vit 
f=1—-——-—=—1— 
2s? # 


On the other hand, s? is the “within trials” mean square, which 
may be expressed 


1 
’= eg [Vi + (kK—1) Viz]. 


Hence 
kV iz oe Vir Viz poe 
Vit (k—DVa. Vit &—DVis 

12. Maximum likelihood estimates: k trials 

In this section we shall indicate in brief outline how the results 
of Jackson mentioned in section 7 may be generalized to the case of k 
trials, where k > 2. We require two assumptions in addition to those 
_ of section 4: 


(a) The population value, o?, of the variance on a particular trial 
is constant over trials. 

(b) The population value, p, of the correlation between any pair 
of trials is constant for all pairs of trials. 


With these assumptions, the probability density function of the 
set of nk variables Xj, is 


* 








7T=1 


(17) 





Bp—A(1— (k—2)p) 


ee ae (1 + (é—1)p) 
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where 
K = [(22)* o*(1— p)**(1 + (K—1)p)]-™, 
A= (Xin—Mi)?, B=2Z ZT (Xjp—My) (Xig—My), 
jh i,peq 
and M, = the population mean for the h“ trial. 

The maximum likelihood estimates of M,, M,, p, and o are ob- 
tained by differentiating the function p with respect to these four 
variables, then setting the derivatives equal to zero, and solving for 
the quantities in question. The procedure is completely analogous to 
that exhibited in Jackson (6, 107-112) and need not be given in de- 
tail here. We find the following maximum likelihood estimates: 


A 1 A A P B 
M; =- X,); #=—, p»=——-_, 
Sa (2 mys nk , 


A(k—1) 
where A and B have the same form as above, except that in them each 
M has been replaced by the corresponding estimate M. On the other 
hand, it may be shown algebraically that A and B can be expressed in 
terms of the variance estimates as follows: 
A=k(n—1)(s?+82), B=k(k—1) (n—1) 82, 

where s;* and s.”? have the same meaning as in section 4. Thus we have 

‘ B 8;? 

p= ns = 

A (k—1) 8?7+8,? 

where 7* is the intraclass correlation adjusted for trend as defined in 


section 5 (d). 
If to the assumptions (a) and (b) above we add a third, namely: 





(c) The population values of the trial means are all equal 
to M, 
then the probability density function is altered merely to the extent 
of substituting M for M,,M,, and M,. Proceeding as before, we find 
the maximum likelihood estimates 


A 


B 


A 1 A 
M=—)D Xn, o= 


A a 
nk i,h nk ‘ 


A(k—1) 
where now A and B have the values 
A= (n—1)Vi t+ n(k—1)Voi, B= (k—1)[(n—1)Vi—2Veui] , 
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so that 


oe (n—1)Vi—2 Vwi 
(n—1)V; + 2(k—1) Vwi 


This is the expression which has been referred to as the symmetrical 
table estimate of the intraclass correlation, section 5(b). It is clearly 
preferable to use the corresponding unbiased estimate. 
13. Program for the estimation of reliability 

We may sum up the essential results of the preceding develop- 
ments in the following outline of the procedure for the estimation of 
reliability when several trials are available: 
(a) Test for trend. We need to test for linear as well as for ran- 
dom trend, and consequently the general test of section 9 must be 
used. 
(b) If no trend of any kind is present, then the unbiased estimate 
of the intraclass correlation is the appropriate estimate [section 
5(c)]. ‘ 
(c) If there is a random fluctuation but no linear trend, then the 
intraclass correlation adjusted for trend [section 5(d)] should be used 
to estimate reliability. 
(d) If linear trend is present, then the reliability may be estimated 
by the use of equation (16), where V, is the error term in an analy- 
sis including a linear term, as outlined in section 9. 
(e) Any one of the above methods of estimation may be seriously 
in error if there is significant heterogeneity of any of the variance 
estimates, either from trial to trial or from individual to individual. 
Homogeneity of variance may be investigated by means of tests de- 
vised by Bartlett (8, 249-251) and by Welch (5, 40-41). 
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AN EMPIRICAL COMPARISON OF METHODS OF 
COMMUNALITY ESTIMATION 


FRANCIS F. MEDLAND* 
THE UNIVERSITY OF CHICAGO 


This paper shows the formulation of nine methods of estimatin 
the unknown communalities. Each of these methods has been us 
on experimental data and the results tabulated for comparison. The 
results show that the most accurate approximations are obtained 
from the Centroid No. 1 and the Graphical methods. 


Communality is defined as that part of the total variance of a 
test which is due to the common factors. Thus, it differs from the 
reliability, which is that part of the total variance that is due to the 
specific factor as well as the common factors. Hence the communality 
is always less than the reliability unless a specific factor is absent. 
In such a case, the values are identical. 

It is customary to record the reliabilities in the diagonal cells 
of a table of intercorrelations. In factor analysis, since the interest 
is centered in the common factor space, it is necessary to use the com- 
munalities as the diagonal entries. Thus it is incorrect to record the 
reliabilities in the diagonal cells unless the variables have been so 
chosen that they contain no specific factors. However, since commu- 
nalities are unknown values to be finally determined by factor analy- 
sis, they must be estimated. 

Fortunately, the estimates of the communalities need not be ex- 
act when the number of tests is large. When the number of tests or 
variables is as small as 10 or 12, it becomes essential to ascertain 
these values with some degree of exactness. In the case of a small 
number of variables, it is customary to determine the communalities 
by factoring the correlation matrix. These communality values are 
then inserted in the diagonal cells and the matrix is factored again. 
When the communalities have been determined in this manner, the 
diagonal elements are not re-estimated for each successive residual 
table. The purpose of this procedure is to minimize the effect of error 
variance on the resulting factor matrix. On large batteries, the com- 

* The author wishes to express his appreciation to Professor L. L. Thurstone 


for his advice and for providing the facilities of the Psychometric Laboratory for 
this investigation. 
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putational labor involved makes this method prohibitive. Thus it has 
become common practice to insert the highest correlation coefficient 
in a particular column as the diagonal element in that column. When 
this is done, a new set of diagonal elements is determined for each 
residual table. 

Recently, Holzinger (2) and Thurstone (3) have developed fac- 
toring methods that eliminate the necessity of computing a residual 
table after each factor has been extracted. For the application of 
these methods, accurate estimation of the communalities is required. 
With these applications in mind, several methods have been devel- 
oped (1, 5). 

The present study is an analysis of the comparative efficiency 
of the various methods of estimation. For purposes of the compari- 
son, a sixty-three variable correlation matrix has been used (4). 
Since the factor matrix for this battery is available, the communal- 
ities that have been determined by factoring are known. These val- 
ues are used as the criterion. Thus for each method here developed, 
the sixty-three estimated communalities have been obtained. The re- 
sults and an analysis of the relative accuracy of the various methods 
are presented. 












Methods of Estimation 





a. Graphical Method 
In two nearly collinear tests (1 and 2) the correlations of their 







h 
respective columns should be proportional to the ratio — or 
2 






12; Reh; COS de; 
soso a) 


Nang . ’ 
T,;  h,h; cos ¢,; 







and, since the cosines are nearly the same, 






(2) 







With test 1 and test 2 (2 is the test which correlates highest with 
test 1) as coordinates, the correlations of the other variables with 
test 1 are plotted against their corresponding correlations with test 
2. This should give a linear plot, so that a line, of slope s, can be 
passed through these points and the origin by inspection. The slope, 










h. 
8,,, of this line is the estimated ratio, “< so that 


Y25 = Tis 
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and since 7,. should be nearly equal to h,h. it becomes: 


Pe Tr % i 
h, — 3 and h,? = 12 S21, (4) 
21 


by which the communalities can be estimated. 

In the six methods next considered, the solution is determined by 
the use of sub-groups chosen from the correlation table. In choosing 
sub-groups, those variables are selected which correlate highest with 
that variable whose communality is to be determined. Usually three 
to five variables are chosen. If any coefficient in the sub-group is dis- 
proportionately low, that variable is eliminated from the sub-group. 


b. Sub-Group Formulas 

1. Centroid No. 1. The communality to be determined is de- 
noted h,?. Tests 2,3,---,s., are the other variables in the sub-group. 

As a first estimate, the highest coefficient in each column of the 
sub-group is recorded. The sum of the first column, then,is (>, + t:), 
where 7; is the sum of all of the known coefficients in the first col- 
umn and ¢, is the highest coefficient in that column. The sum of all 
of the coefficients in the sub-group is (Sv + St), where Sr is the sum 
of all the known coefficients and St is the sum of all of the diagonal 
estimates. 





Then: 
Sr, + f, 
hg Se eersenrnes (5) 
Vor + dt 
or 
(S17, + t,)? 
h,? = en . (6) 
Sr + St 


2. Centroid No. 2. The communality to be determined is de- 
noted by h,* and tests 2, 3, --- , s, are the other variables in the 
sub-group. The communality of test 1 is estimated by the test pro- 
jection on the centroid axis of the sub-group. This will, in general, 
be an underestimation unless the test vector should coincide with the 
centroid axis. To compensate for this underestimation, the commu- 
nalities of the adjacent tests are underestimated by entering the aver- 
age of the adjacent columns in their diagonals. The sum of adjacent 


s(Sr— 31) 





columns 2, 3, ---, 8, becomes — where Sv is the sum of 
8 —_— 


all of the known coefficients in the table and S17, is the sum of all of 
the known coefficients in the first column. 
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The sum of all of the coefficients in the table is estimated as 





(or, +h) + 8(Sr— =n) 
(s—1) 
where the complete sum for the first column becomes ! 
Sr, + a3. 


The projection of this test vector on the centroid axis is the ratio 
of the first column sum to the square root of the sum of all of the co- 
efficients in the table. This projection should be an estimate of h,?. 


Then 
(Sr, + &,*)* 
h?= ; (7) 
s(S7r~ 37) 
Dn + hy? + 


(s—1) 








which simplifies to: 
in, oe ia 
s>r— (28—1) 57, 
For routine computing an equivalent form is developed: 
__ @-1) Cn) 
Sr, +28 D1’ 
where s is the number of tests in the sub-group, 57, is the sum of all 


of the known coefficients.in column 1, and >7, is the sum of all of the 
adjacent coefficients below the diagonal. 

3. Three-Test Formula. If a cluster of three highly correlated 
tests is chosen, these test vectors will be separated by very small an- 
gles, so that 





(9) 





Tj 
hte (10) 


T jk 





4. Four-Test Formule. Expanding the equation (10) to the case 
of four correlated tests, 


hy Tie kT M15 Tu 





T jk Tk Tit 


Combining these terms and using the geometric mean as an ap- 
proximation, it becomes: 


T,;* Tk? T;? 
h,? = ,| ——_————. (11) 
Vik Vit Tre 
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5. Summation Formula. Let test 1 be the variable the commu- 
nality of which is to be determined, and let test 2 be that variable 
which correlates highest with it. If these two tests have small an- 
gular separations, their intercorrelations should be nearly propor- 
tional, so that 

h? 11 — 12 
i ae, (12) 
2 Le — Te 
where >, and 572 are the sums of the known coefficients in columns 
1 and 2. From this the computing formula is derived: 


mas N12 (S11 — 112) 

° (Sre— rz) ‘ 

6. Spearman Formula. A generalization of equation (13) can 

be made for a group of four correlated tests which may be assumed 

to be nearly collinear. From equation (13), it becomes for the four 

tests 1, 2, 3, 4, 
sista Ty2 (S71 — Nz) a T13 (571 — 113) ‘al Tia (Si — Tua) 
re — Nr2 1s — 118 p> ae 

where >7,, >r2, 5’:, D7., are the sums of the known coefficients in 

columns 1, 2, 3, 4, respectively. From equation (14) 


mete N2(S1r1 — 2) + is(S%1 — M13) + Nua (S01 — Nu) 





(13) 





» (14) 








| 








“iY ? (15) 
(Sre— 1:2) + (Ss — 118) + (US — Nu) 
which simplifies to the computing form 
(S7,)? — (S17) 
h?= , (16) 


=r—257 


in which Sr is the sum of all of the known coefficients in the sub- 
group, and (Sr,’) is the sum of the squares of all of the coefficients 
in the first column of the sub-group. 

7. Highest Coefficient in the Column. From equation (10) a 
simplified estimate may be derived: 


A= 135 Vik 

Vik 

If these coefficients are of the same order of magnitude, the esti- 
mated communality then approaches the highest intercorrelation in 
the column. While this method is generally applicable to large corre- 





106 PSYCHOMETRIKA 


lation tables, it is, ordinarily, too unstable for use on small correla- 
tion matrices. 

8. Carlson’s Method. This procedure (1) has been developed in 
connection withh a simple multiple-factor approximation method. 

This graphical method is generalized for use when more than 
one factor is present in the correlation matrix. Let test 1 be the test 
for which the communality is to be determined, and test 2 that test 
which correlates highest with test 1. 

On cross-section paper, an ordinate (test 1) is erected with val- 
ues from .00 to 1.00 and, at an arbitrary distance of one unit, a sec- 
ond ordinate (test 2) is erected. The correlation coefficients of test 
1 with each of the other tests are indicated on ordinate 1, and the 
correlation coefficients of test 2 with each of the other tests are in- 
dicated on ordinate 2. Lines connecting the paired values 740720 , 10720 , 

- , Tino, , are extrapolated to the abscissa. Paired values that are 
unlike in sign or paired values which are lower on the first ordinate 
than on the second ordinate are ignored. 

The distance on the abscissa from test 1, with the distance be- 
tween ordinates 1 and 2 as the unit, to the point where each of the 
remaining lines crosses the abscissa is measured. The sum of these 
distances divided by the number of tests, plus 1, gives the point on the 
abscissa from which the communality can be determined. From this 
point a line is drawn through 7,. on ordinate 2 and extrapolated to 
ordinate 1. This intersection on the first ordinate is the communality 


estimate for test 1. 


Results 


By using each of the methods outlined, communalities were esti- 
mated for each variable of the sixty-three test battery. A frequency 
distribution of absolute deviations from criterion values (Table 1) 
shows that the Centroid No. 1 and the graphical methods give results 
that are most consistent with the criterion. “Root mean square” 
deviations computed on these data show no significant difference be- 
tween these two methods on the basis of accuracy. However, the 
amount of labor involved places the more favorable emphasis upon 
the Centroid No. 1 method. A distribution of the algebraic devia- 
tions from the criterion (Table 2) is used to determine whether a 
particular method gives consistent overestimation or underestima- 
tion of the true values. Inspection of these distributions and their 
means does not give significantly different results among the various 
methods. Further comparison of these tables shows that those meth- 
ods which depend upon a single correlation coefficient give the least 
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stable results. This is due to the error in the single correlation. co- 
efficient upon which these formulas pivot. The methods that are sub- 
ject to this criticism are highest coefficient in the column, three-test 
formula, and summation formula. 

Two follow-up studies on smaller batteries (21 variables and 15 
variables) show the same results with the exception of “highest co- 
efficient in the column,” which proved too unstable for use on the 
smaller tables. 
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TABLE 1 
Absolute Deviation from Communality Obtained by Factoring 
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PSYCHOMETRIKA—VOL. 12, NO. 2 
JUNE, 1947 : 


TABLES FOR THE DETERMINATION OF THE SIGNIFICANCE 
OF SKEWNESS AND OF THE SIGNIFICANCE OF THE 
DIFFERENCE IN THE SKEWNESS OF TWO 
INDEPENDENT DISTRIBUTIONS 


ERWIN K. TAYLOR* 
PERSONNEL RESEARCH SECTION, AGO. 


Several of the more popular statistics texts used in psychology 
are reviewed with respect to their treatment of the measurement and 
interpretation of skewness. Some areas in psychology where meas- 
ures of skewness of distributions may yield significant information 
are indicated. Tables of 1%, 2% and 5% points of moment measures 
of skewness and tables of the 1% and 5% points of the difference in 
skewness of two uncorrelated distributions are presented. These 
tables are approximations based on the approximate normality of 
skewness in large samples from normal populations. The limited 
confidence with which these tables can’ be used in the absence of ex- 
act knowledge of the distribution function of the underlying statis- 
tics is indicated. 


It is the purpose of this paper (1) to compare several popular 
statistical texts in their treatment of skewness, (2) to indicate areas 
in psychology in which measures of skewness may be important and 
(3) to present tables of the 1%, 2% and 5% points of skewness and 
of the 1% and 5% points of the difference in the skewness of two un- 
correlated distributions. 

It is an almost universal practice in psychological studies involv- 
ing frequency distributions to present the mean and standard devia- 
tion of each such distribution. It is almost as universal to omit meas- 
ures of symmetry. This omission is a reflection of (or possibly is re- 
flected in) the statistical texts usually used by psychologists for both 
research and teaching. Confining ourselves for the purpose of this pa- 
per to the discussion of measures of symmetry and their use, we might 
do well to examine several of these texts. Lindquist devotes one para- 
graph to skewness in his elementary book (13). All that appears in this 
paragraph is a definition. No method of measuring skewness is given. 
In his Statistical Analysis, Lindquist (12) makes no mention of the 
phenomenon insofar as a perusal of the index indicates. Edwards (4) 


* The opinions expressed in this article are those of the author and are not 
to be interpreted as official War Department policy. 
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presents a section titled “Skewed Distributions” in which he defines 
both skewness and kurtosis. His last sentence reads, “These are meas- 
ures of skewness and kurtosis, but we shall have little need for them 
and they are not included here.” 
; _ 8(M—Mdn) 
Peters and Van Voorhis (16) give as one of the 


Co 
best measures of skewness. f, is mentioned. No mention is made of 


: M — Md 
the standard error of “ n) . No formula for the standard 


Cc 
error of {, is given; reference, however, is made to sources where it 
may be found. Aside from the statement that £, is equal to zero for 
a normal distribution, wast further is said of skewness or its im- 
plications. 

Garrett (8) says, “It i is often important to know (1) whether the 
skewness which so often occurs in distributions of test scores and 
other measures represents a real divergence from the ideal normal 
curve; or (2) whether it is the result of chance fluctuation, arising 
from temporary causes, and is not significant of real disagreement.” 
Why it is important to know whether the skewness is significant or 
not, the author does not say. Nor does he tell his reader what inter- 
pretations are to be made in the event that there is a significant depar- 











(Poo + P..) 
ture from symmetry. He recommends ses” “rete — 50 as a measure 
5185 (Po. — Pro 
of skewness and gives its standard error as ( —_ ae) , assum- 
VN 


ing normal] distribution of this function. 

Guilford in both Psychometric Methods (9) and in Fundamental 
Statistics (10) goes somewhat more fully into the matter. In the 
former volume (p. 105) he says, 


There are a number of possible causes for skewed distribu- 
tions. One cause is a selective factor, or set of factors, which 
cause the measurements to deviate from a random sampling. 
If in testing the intelligence of fourteen-year-old boys we 
used only those below the ninth grade, we would probably 
find a distribution that is positively skewed because the 
brighter fourteen-year-old boys in the high-school grades had 
been eliminated. Another frequent cause in mental testing 
is the use of a test that is too difficult or too easy for the 
group to which it is applied . . . Another cause may be the 
measuring instrument. If the units of amount near the lower 
end of the scale are in reality larger than those at the up- 
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per end, there will be an apparent bunching of cases at the 
lower end and therefore a positive skewing effect. If we 
eliminate all these factors causing artificial skewing, and 
other such factors as well, the skewness may express the ac- 
tual facts. The distribution may in reality be one lacking in 
symmetry. Should this be distinctly so, the application of the 
normal curve is questionable and one of Pearson’s other types 
should be used. 


In addition to the importance of skewness in the fields of mental 
and educational testing pointed out by Guilford, investigators in other 
areas of psychology have employed departures from symmetry as 
highly meaningful characteristics of their measures or populations. 
Katzoff (11) reworking Allport and Solomon’s (1) data found skew- 
ness a much more satisfactory index of conforming behavior than 
was the “J-curve” hypothesis proposed in the original research. Both 
Yoder (20) and Ford (7) indicate the possible influence of restriction 
of output upon the symmetry of production curves, presenting some 
data to indicate negative skewness under such circumstances. Bliss 
(2) found that artificial restriction of output resulted in negatively 
skewed distributions of production, while poor motivation resulted in 
positively skewed distributions. The apparent contradiction between 
the findings of Bliss and those of Yoder and Ford vanishes upon analy- 
sis. In the case of the latter writers, the situation was one in which 
there was high motivation to reach a given point and fear of rate cut 
if that point was exceeded. This would naturally result in a negatively 
skewed distribution. Bliss, too, secured such a distribution when the 
output of his subjects was artificially restricted beyond a certain point 
by the speed of the machines on which they worked. 

The writer in some unpublished research in a sales organization 
found that the production (sales) distribution shifted from a posi- 
tively to a negatively skewed curve when high motivation in the form 
of well conducted sales contests was introduced into the situation. In 
the case of individual salesmen, those whose distributions of sales be- 
fore the contest were positively skewed generally showed greater gain 
than those with similar means but negatively skewed or symmetrical 
distributions. 

An inspection of Uhrbrock’s (19) data on the attitude of factory 
workers, clerks, and foremen reveals the extent to which departures 
from symmetry are related to attitudinal changes. Another psycho- 
logical field in which skewness may have important implications is 
that of learning. While the writer can cite no evidence to support this 
contention, it seems logical that for any given individual who has 
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learned a particular skill completely, the distribution of the accuracy 
or of the speed (where not artificially restricted by zero time) with 
which repeated trials are accomplished should be symmetric about the 
mean accuracy or speed of the individual. Motivation in this case 
should of course be ruled out. The novice on whom the distribution 
of repeated trails is available, it is hypothecated, would be inclined to 
present a positively skewed distribution. As learning progressed, the 
distribution should approach symmetry as a limit. If this relation- 
ship could be empirically demonstrated, it might prove quite useful 
in many learning experiments. 

It is apparent that there are several areas in psychology in which 
the significance of the departure of distributions from symmetry may 
be of considerable importance. By the same token there are situations 
in psychological research in which it is desirable to determine wheth- 
er the skewness of one distribution is greater than that of another. 
As in the case of all statistics, such determinations must be made in 
terms of the standard error of the measures employed. 


Of the many measures of skewness, a; = \/, or Fisher’s* (6) 
9: = [N?/(N —1) (N—2)] as (1) 


are probably most satisfactory from the point of view of both statis- 
tical efficiency and computational ease. A method for the systematic 
computation of a; is given by Cureton and Dunlap (3). a; may of 
course be easily converted to g, as indicated above. The systematic 
computation of g, is given by Snedecor (17). 

Having determined the value of skewness of a given distribution, 
it becomes necessary to determine whether or not such skewness is 
significantly different from zero. Unfortunately, the distribution func- 
tion of neither of the recommended measures of skewness has been 
determined. The basic work of Fisher (5) has established the vari- 
ance of g, (and necessarily of a,;) from normally distributed popu- 


lations as 


* This is predicated on the computation of o by the formula 


ae J =(X —X)? 
(N—1) © 

a — X)2 
N 

is used in the computation of o, the relationship between a, and g, becomes 


N—1\% N2 
1=( N ) (a—pacn)™ 


If 
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6N (N —1) 
c= 5 (2) 
% (N—2)(N+1)(N +8) 

Theoretical as well as empirical evidence tends to indicate that 
the distribution of g, (or a3) is approximately normal for samples 
drawn from a normal population. Pepper (15) drew 700 samples of 
size 10 from a normal population. The distribution of the measures 
of a; was tested for departure from normality by chi squared. It was 
found that departures from normality were less than would be found 
by chance alone 97% of the time. So good a fit indicates the feasibility 
of using normal curve area tables to test the probability of an ob- 
tained a; (or g,) having arisen in a sample from a symmetrical uni- 
verse. 

Pearson (14), on the basis of the work of Fisher (5), published 
tables of the 1% and 5% points of a; for:samples of size 50 to 5000. 
These were extended by Williams (18) to samples of size 25. These 
tables are based on the formula 


oa, — V6/N ° 


Table 1 presents the 1%, 2%, and 5% points of both a; and g, on sam- © 
ples of the same range of N but using the more exact formula 


-| 6N (N —1) " 
"| (N—2)(N +1) (N +3) 


In addition, Table 1 gives the variance and standard error of g: as 
well as the constants by means of which a, may be converted to g, and 
g, to a;. This table may be used to determine the significance of de- 
parture of skewness from zero at any of three levels. 

Table 2 presents the 1% points of the difference in skewness (as 
measured by g:) of two uncorrelated distributions of size from N = 30 
to N = 5000 for either. Table 3 presents the 5% points of the same 
statistics. In using these tables it should be noted that these points 
are based on the formula 


Vo, +o, °K, (4) 








(3) 


where 
K = 2.58 for Table 2 and 
K= 1.96 for Table 3. 


This neglects the (27 0, o,,) term common to formulas for the stand- 


ard error of the difference between two statistics. These tables should 
be used only in connection with uncorrelated variables. 
Where it is required to determine the significance of the difference 
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between correlated skewness measures and that correlation is known, 
the proper formula may be easily derived from the o*, or o*<, columns 


of Table 1. The needed statistic is 





Caift => Vo’, = te a, — 2, g. Gg Gg -« (5) 
le lp la 1p la 1b 


In attempting to determine the significance of the difference 
in skewness of two samples drawn from non-normal populations, 
the writer has been unable to find any evidence with respect to 
the distribution of such differences. Undoubtedly considerable re- 
search into the question is required before Tables 2 and 3 can be used 
with complete confidence. The assumption of normality without proof 
thereof is not without considerable precedent in the field of statistics. 
In general, it appears that the statistics from non-normal universes 
are more nearly normal than are their parent populations and that this 
tendency, toward the Gaussian curve is a function of N. Until the 
distribution functions of the statistics under consideration have been 
developed, the tables presented can probably be used in practice with- 
out doing extensive violence to the data. 
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TABLE 1 
Standard Errors and Fiducial Limits of Skewness 
2 (N—1) (N—B) 
1)N 2) 0% 3 4 5) ——————_— (6) 52 L. g. 
(1) (2) 029, (3) og, (4) (N—-1) (N—2) (5) Ni 1 
5000 001199 084626 1.00060 999400 .067866 
4000 001499 038716 1.00075 999250 .075882 
— 8000 .001998 044698 1.00100 .999000 087606 
2000 002995 054726 1.00150 998500 107261 
1500 .003992 .063182 1.00200 998001 123824 
1000 005982 077348 1.00301 997002 151589 
900 006644 081511 1.00334 996669 159758 
800 .007472 086441 1.00376 996253 169421 
700 008534 092379 1.00430 .995718 181059 
600 .009950 .099749 1.00502 995006 -195504 
500 011928 .109215 1.00603 994008 214057 
450 013245 115086 1.00670 993343 225564 
400 .014888 .122016 1.00754 992512 .239146 
350 .016997 3130369 1.00863 991445 255518 
800 .019802 140719 1.01008 990022 275804 
250 .023716 154000 1.01211 988032 301848 
200 -029558 -171924 1.01518 -985050 .8386964 
175 033711 .183605 1.01737 982922 859858 
150 032919 198037 1.02082 .980089 888145 
125 046881 .216520 1.02446 .976128 424871 
100 058264 241379 1.03072 .970200 473098 
90 064532 254031 1.08422 966914 497891 
80 072312 268909 1.03862 962812 527051 
10 082225 .286749 1.04433 957551 .562017 
60 095292 308694 1.05202 .950556 605028 
50 .113300 .336600 1.06293 .940800 659723 
45 125126 353781 1.07030 984821 693299 
40 139714 873783 1.07962 926250 .732600 
35 .158160 897693 1.09180 915918 .779462 
30 .182237 426892 1.10837 902222 872994 
25 215002 463683 1.13225 883200 957042 














TABLE 1 (continued) 
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N (7) 2%L.g, (8)1%L.g, (9) 5% Lo, (10) 2%L.o, (11) 1% La, 
5000 080552 089190 067825 080504 089136 
4000 090067 099725 0715825 089999 099650 
3000 108983 115134 087184 108879 .115019 
2000 127811 140964 .107100 .127120 .140753 
1500 .146983 162745 123586 146689 162420 
1000 .179926 199222 151135 179887 198625 

900 189622 209958 159226 .188990 209259 
800 201091 222656 .168786 200338 221822 
700 214905 237952 .180284 213985 236983 
600 232050 256935 194528 230891 255652 
500 254071 281318 212774 252549 279682 
450 267729 296441 224062 265947 .294468 
400 283851 314291 237355 281726 311988 
350 308283 336171 253332 300688 838295 
300 827360 362467 273052 324094 358850 
250 858256 396676 298222 353968 391929 
200 899954 442845 331926 398975 436224 

175 427128 472988 858712 419834 464856 

150 460701 510108 380417 451528 499951 

125 508699 557717 414240 491675 544403 

100 561530 621749 458995 544796 603221 

90 590962 654338 481418 571409 682869 

80 625574 692661 507451 602310 666902 

70 667076 .738614 538160 688759 .707261 

60 718127 .795140 575113 682620 755825 

50 783046 867021 620667 .736690 815693 

45 822899 911147 647764 768852 851304 

40 869546 962798 678571 805602 891792 

35 .925169 1.024386 -713923 .847379 .938254 

30 1.051008 1.176514 187634 948243 1.061477 

25 1.155498 1.296921 1845259 1.020536 1.145441 
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TABLE 2* 
1% Limits of 9,, — 9iy 
N 5000 4000 3000 2000 1500 1000 900 
5000 1261 
4000 1838 1410 
3000 1456 1523 1628 
2000 1663 1722 1816 1986 
1500 1856 1909 1994 2149 2302 =. 
1000 2183 2228 2301 2437 2572 2817 
900 2281 2324 2395 2526 2656 2894 2969 
800 2399 2440 2507 2632 2758 2988 3060 
700 2533 2572 2635 2756 2876 3097 3167 
600 2720 2756 2816 2928 3041 3251 3318 
500 2951 2985 8040 3144 3250 3447 3510 
450 3096 3128 3180 3280 3382 3572 3683 
400 3267 3297 3347 3442 3539 3721 3780 
350 3475 3573 3550 3640 3752 3905 3960 
300 3733 3759 3803 3887 3973 4136 4189 
250 4066 4090 4130 4208 4288 4439 4488 
200 4517 4539 4576 4646 4718 4856 4901 
175 4813 4833 4867 4933 5001 5182 5174 
150 5178 5198 5229 5291 5354 5476 5516 
125 5648 5666 5695 5751 5810 5922 5959 
100 6281 6297 6323 6374 6427 6529 6562 
90 6604 6619 6644 6692 6743 6840 6872 
80 6984 6998 7022 7067 7115 7207 7238 
70 7440 7453 7475 7518 7563 7650 7679 
60 8001 8014 8084 8074 8116 8197 8224 
50 8716 8727 8746 8783 8821 8896 8921 
45 9155 9166 9184 9219 9256 9327 9850 
40 9669 9679 9697 9730 9765 9832 9854 
35 10283 10292 §=10808 10340 10372 10436 10457 
30 8 §=©.11032 11041 11056 11085 11116 11175 =: 111195 





* Entries in Tables 2 and 8 have been multiplied by 104. 
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TABLE 2 (continued) 
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11219 





11249 





N 800 700 600 500 450 400 350 300 
5000 
4000 
3000 
2000 
1500 
1000 
900 
800 3148 
700 3253 3355 
600 3400 3496 3634 
500 8693588 3679 3810 3978 
450 3707 3796 3923 4087 4192 
400 3852 3937 4060 4218 4820 4445 
350 4029 4110 4228 4381 4479 4599 4749 
300 4254 4831 4448 4588 4683 4789 4941 5126 
250 4549 4621 4726 4863 4952 5061 5197 5878 
200 4957 5023 5120 5246 5329 5480 5558 5723 
175 5227 5290 5382 5503 5582 5678 5800 5959 
150 5566 5625 5712 5825 5900 5992 6107 6258 
125 6005 6060 6141 6247 6316 6402 6510 6652 
100 6604 6654 6727 6824 6888 6967 7066 17197 
90 6912 6960 7080 7123 7184 7259 7355 17480 
80 7276 7321 7388 7476 7534 7606 7698 7818 
70 7714 7757 7820 7904 7959 8027 8114 8228 
60 8257 8297 8356 8434 8486 8550 8631 8789 
50 468952 8698989 90438 9115 9163 9222 9298 9397 
45 9380 9415 9467 9536 9582 9638 9711 9806 
40 9882 9916 9965 10031 10074 10128 10197 10288 
35 10483 10515 10561 10623 10664 10715 10780 10866 
30 11292 11350 113889 11436 11497 11578 
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TABLE 2 (continued) 









































250 200 175 150 125 100 90 
5610 

5945 6263 

6173 6479 6688 

6462 6755 6956 7214 

6844 7122 7312 7558 7887 

7375 7638 7812 8042 8352 8793 

7652 7901 8074 8297 8598 9026 9254 
7982 8221 8387 8602 8893 9808 9529 
8384 8612 8771 8976 9255 9655 9868 
8886 9101 9252 9447 9712 10094 10298 
9585 9736 9876 10060 10309 10669 10862 
9988 10131 10266 10442 10683 11031 11218 
10418 10598 10727 10896 11127 11461 11641 
10985 11160 11283 11444 11664 11983 12155 
11690 11854 11970 12122 12829 12682 12796 

TABLE 2 (continued) 
80 70 60 45 40 85 30 
9796 
10126 10446 
10545 10858 11245 
11097 11890 11764 12262 
11445 11729 12098 12577 12886 
11861 12185 12487 12957 18256 13616 
12366 12629 12968 18420 18710 14058 14487 
18246 14280 14615 15028 15551 
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TABLE 3 


5% Limits of 9,, — Diy 
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TABLE 3 (continued) 


























N 800 700 600 500 450 400 350 8 8§66300 
5000 
4000 
3000 
2000 
1500 
1000 
900 
800 2395 
700 2475 2553 
600 2587 2660 2765 
500 2730 2799 2899 3027 
450 2821 2888 2985 3110 3190 
400 2931 2996 3089 3210 3287 3382 
350 3066 3127 3217 3334 3408 3499 3614 
300 3237 3296 3381 3491 3563 3651 3760 3900 
250 3461 3516 3596 3700 3768 3851 3954 4088 
200 3772 3822 3896 3992 4055 4132 4229 4355 
175 3977 4025 4095 4187 4247 4820 4413 4534 
150 4235 4280 4346 4432 4489 4559 4647 4762 
125 4569 4611 4673 4753 4806 4871 4954 5062 
100 5025 5063 5119 5192 5241 5301 5377 5476 
90 5259 5296 5349 5420 5466 5523 5596 5692 
80 5536 5571 5622 5689 5733 5787 5857 5949 
70 5870 5902 5950 6014 6056 6108 6174 6261 
60 6283 6313 6358 6418 6457 6506 6567 6650 
50 46812 6840 6881 6936 6972 7017 7075 7150 
45 71387 7164 7204 7256 7291 73834 7389 7461 
40 7519 7545 7582 7633 7665 7706 ‘7759 7828 
35 7977 8001 8086 8083 8114 8153 8208 8286 
30 8537 8559 8592 8636 8666 8702 98748 8810 
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TABLE 3 (continued) 








































10866 





11121 





11435 





N20 200 175 150 125 100 90 

250 4269 

200 4524 4766 

175 4697 4930 5089 

150 4917 5140 5298 5489 

125 5208 5419 5564 5751 6001 

100 5612 5808 5944 6119 6355 6691 

90 5822 6012 6144 6818 6542 6868 7041 

80 6074 6255 382 6545 6767 7088 7251 

70 6879 6553 6674 6880 7042 7347 7509 

60 6761 6925 #7040 7188 7390 7681 7836 

50 7255 7408 +7515 7655 ‘7844 8118 8265 

45 7562 7709 7811 7945 8129 #98394 8536 

40 7928 8064 8162 8291 8467 8721 8858 

35 8359 8492 8585 8708 8875 9118 9249 

30 8895 9020 9108 9224 9881 9612 9787 
TABLE 3 (continued) 

80 2~—0 60 50 45 40 35 30 

7454 

7105 7948 

8024 8258 8556 

8444 8667 8951 9880 

8709 8925 9202 9570 9805 

9025 9243 9501 9859 10087 10861 

9409 9610 9867 10211 10432 10697 11028 
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A MODIFIED AITKEN PIVOTAL CONDENSATION METHOD 
FOR PARTIAL REGRESSION AND 
MULTIPLE CORRELATION 


ALICE VAN BOVEN 
REDLANDS, CALIFORNIA 


In this modification of the Aitken Pivotal Condensation Method 
for obtaining partial regression weights and multiple correlation 
coefficients, the unit matrices used by Aitken are eliminated, with a 
resultant decrease in the computaticnal work. Through a simplified 
arrangement of successive matrices, each one is reduced by the same 
simple rule. Other modifications also contribute to ease of compu- 
tation and facilitate checking. 


The method of using determinants to solve a system of simultane- 
ous equations has long been accepted. An excellent comprehensive sur- 
vey of a number of such methods can be found in an article by P. S. 
Dwyer in Psychometrika, 1941, 6, 101-129. A partial regression equa- 
tion for three or more variables is constructed by solving as many 
simultaneous equations as there are variables. Each of these equations 
is derived mathematically from the assumption that the variance of 
the error of estimate is to be reduced to a minimum when each com- 
ponent in turn is permitted to vary while all the others are held con- 
stant. 

Godfrey Thomson in his book The Factorial Analysis of Human 
Ability* describes the Aitken Pivotal Condensation Method for obtain- 
ing the regresesion weights and multiple R by employing a series of 
matrices. An example which employs the same general technique and 
nearly the same steps is worked out in an article by Godfrey Thomson 
in the Journal of Educational Psychology, 1936, 27, 51. Both of these 
examples of the Aitken method start with a matrix of first-order cor- 
relations and a unit matrix beside it. These two matrices are gradually 
condensed until the beta coefficients are obtained. Two Fisher methods 
employing unit matrices are shown by Dwyer on p. 122 of the article 
referred to above. 

In the method here described, the unit matrices employed by Ait- 
ken are eliminated. The many zeroes in the unit matrices are not need- 
ed to obtain the final results. The total number of arithmetical calcu- 


* Thomson, Godfrey. The factorial analysis of human ability, Boston: Hough- 
ton Mifflin, 1939, pp. 89-95. 
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lations is thus materially reduced. A further contribution toward com- 
pactness and ease of computation is found in the series of matrices 
set up at the right-hand side of the computation sheet. (These matrices 
are labeled A’, B’, C’, and D’ in the sample problem shown in Table 
1.) It adds much to the ease of application to have a space provided 
on the computation sheet for the figures derived by multiplication 
which in turn are to be subtracted from numbers in the original ma- 
trix. This multiplication must be done anyway when solving determi- 
nants or matrices set up for the purpose of solving a series of simul- 
taneous equations. Writing them out in matrix form shows the sym- 
metric pattern obtained and also provides the conventional check by 
addition at the end of each row, thus decreasing the labor and the 
chance for errors to slip in at these points. The figures are all on the 
page, so that when rechecking is necessary, time is saved. 

There are in all but the last two matrices (C , D, C’, and D’ in the 
sample) some symmetric numbers, which, as far as the final results 
are obtained, could be omitted from the computation sheet, exactly as 
in several of the examples surveyed by Dwyer. For the sake of the 
check by addition of each row, it is, in the opinion of the author, better 
to copy them on the computation sheet. 

Aitken* describes the use of a pivot and pivotal condensation for 
the solution of determinants as follows: 


A determinant of order being reduced by a first pivot- 
al condensation to one of order n-1, the latter in its turn can 
be reduced by a second pivotal condensation to one of order 
n-2, and so on until finally we have a determinant of order 2, 
a simple cross-product. Naturally the case where the pivot is 
1 or — 1 will give the least arithmetical work. It is always 
possible, by a preliminary division of the row or column con- 
taining any non-zero element, to turn that element into a unit 
pivot. 


The chief advantage, for practical purposes, in the method here 
presented is in the simplicity of arrangement, in that each matrix is 
condensed by the same simple rule which is readily learned. 

A description of a sample application of the modification of Ait- 
ken’s method is presented step by step so that it can be followed by 
comparing the text with the sample problem appended (Table 1). 

Let us say there are n independent variables, labeled V, , V2, Vs 
_..» Vn. There is also the criterion, C. The method consists of n 


* Aitken, A. C. Determinants and matrices, Edinburgh: Oliver and Boyd, 
1989, p. 47. 
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double matrices, arranged successively and solved in the same manner, 
so that once the process is learned, the work is largely routine. 

When laying out the problem on paper, allowance should be made 
for 2n + 2 columns across the page. It is a great help to enter all nega- 
tive values in red throughout the problem. 

Matrix A. This is simply a correlation matrix of the n independ- 
ent variables, with the criterion correlations added on a row below 
and two columns added at the right. The rows and columns of the 
symmetrical square wherein are entered correlation values of the n 
independent variables are labeled V,, V2, V;, --- , Vn, respectively, 
and will henceforth be so designated. The bottom row of the matrix 
in which are entered correlation values between the independent 
variables and the criterion will be designated as C . To the right of the 
V, column is the — 1 column, where — 1 is entered in all the V series 
rows, but 0 is placed in the C row. The last column of Matrix A is 
the total column, designated as 7. In it are entered the algebraic 
sums of each row of the matrix. Note that in the V series rows the 1 
and — 1 will cancel each other in adding, but there is neither a 1 nor 
a—linrowC. 

Matrix A’. The columns in this matrix are labeled V’., V’;,---, 
V’,, and 7’. In the top row of this matrix the numbers .481, .146, 
.118 , and .745 are copied from Matrix A. Note that the .745 copied 
into column 7” is the algebraic sum of the other three copied numbers. 

The symmetrical part of Matrix A’ is in reality a submatrix con- 
taining »—1 arrays. In the example, it includes the nine numbers 
in columns V’,, V’,;, and V’, and in the rows V., V;, and V,. These 
numbers and the numbers in the 7” column are obtained by multi- 
plying each number in the top row of Matrix A’ by the numbers in 
column V, of Matrix A. In the sample problem, when .481 from row 
V., column V, is multiplied by the numbers .481, .146, .118, and .745 
from the top row of Matrix A’ the results are, respectively, .231 , .070, 
.057, and .3858. This row is checked by seeing that the number in 
the T’ column equals the algebraic sum of the preceding numbers. 
When .146 is used as multiplier for the numbers in the top row of 
Matrix A’, the results are .070, .021, .017, and .109. The last num- 
ber equals the algebraic sum of the other three. Likewise when .118 
is used as multiplier with the same four numbers in the top row of 
Matrix A’, the results are .057 , .017 , .014, and .088 . The last named 
value, .088 , equals the sum of the other three. The .490 in row C, 
column V, must also be multiplied by each number in the top row of 
Matrix A’. The results in this case are .236, .072, .058, and .365. 

A single setting of the slide rule is sufficient for the multiplica- 
tion in each row. Each row of each matrix should be checked by ad- 
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TABLE 1 
Sample Problem in Multiple Correlation 
Job Information Test in Military Correspondence Versus Four Other Variables 
(94 cases, all enlisted men) 








V; Ve eg he , eh, PL eS 
Army Army 
Matrix A AGCT Educ. Trng. Exp.-—1 TotalMatrix A’ Total 
AGCT V, 1 481 .146 .118 -1 .745) .481 .146 .118 .745 
Education V, .481 1 134 -.119 -1 .496] .231 .070 .057 .858 
Army Training Vz, .146 .13841 -.102 -1 .178) .070 .021 .017 .109 
Army Experience V,, .118 -.119 -.102 1 -1 -.103| .057 .017 .014 .088 
Test Score C 490 .819 .815 .829 O 1.453] .286 .072 .058 .365 
D .769 «=—«.064 -176 .481 -1 .188) Matrix B’ 





Matrix B 1 .083 -.229 .626 -1 .480) .083 -.229 .626 .480 
.064 .979 —119 .146 -1- .069| .005 -.015 .040 .031 
-.176 -.119 .986 .118 -1 -.191/-015 .040 -.110 -.085 
.083 .248 .271 .490 0 1.088] .007 -.019 .052 .040 


D' 974-104 .106 .064 -1 .038/ Matrix C’ 


Matrix C 1 -107 .109 .066 -1 .068|-.107 .109 .066 .068 
-.104 .946 .228 —176 -1 -.106) .011 -.011 -.007 -.007 
.236 .290 .488 .083 0 1.048/-.025 .026 .016 .016 


D” 985 .239 -.169 -.104 -1 -.099/ Matrix D’ 


Matrix D 1 .256 -.181 -.111 -1 -.036) .256 -.181 -.111 -.036 
315 .412 .067 .236 0 1.032} .081 -.057 -.035 -.011 


D” 3881 .124 .271 .315 1.048 


























B Values 331 .161 .278 .337 
B times r 162 .051 .088 .111 412) P1034 == -64 
TABLE 2 


Proof of the 8 values by Formula 
Vey = By + Bayo + Batis + Baty,» Cte. 























B V;, Vz V; Vv, Vv, B i V.B i VB 4 VB 4 

381 1 481 146 118 331 .159 .048 .089 

161 aati. 4 184 -.119 OTT 161 022 -.019 

.278 146 184 1 —.102 .041 .087 .278 —.028 

337 118 -119 ~-—.102 1 .040 -040 -—.034 837 
r with criterion 490 319 315 829 

Totals 489 817 314 829 
Differences 001 002 001 .000 
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dition throughout the problem, as just described for Matrix A’. Small 
discrepancies in the last decimal place may result from dropping ad- 
ditional decimal places, but if gross discrepancies are found, an error 
has been made in computation and must be corrected before proceed- 
ing further. 

Another check that should be made for each of the matrices 
on the right-hand side of the page is to see that the part of each 
matrix that is supposed to be symmetrical is so. If not, an error has 
been made which of course must be corrected before continuing with 
the problem. 

Obtaining Values for Row D. Row D is an intermediate row be- 
tween Matrix A and Matrix B, and is obtained by subtracting in 
row V. the numbers in Matrix A’ from the corresponding numbers 
in Matrix A . From the 1 in row V., column V, is subtracted the .231 
in row V., column V’;. The resulting .769 is placed in column V, of 
the D row. Continuing the subtractions in row V., from .134 is 
subtracted .070 , and the difference .064 is placed in the D row; from 
—.119 is taken .057, and the algebraic difference, — .176 , occupies 
the next place in the D row. Since there is nothing to subtract from 
the — 1 in row V, , the .481 from the first column of the row is brought 
down into the next column of the D row. A — 1 is entered in the — 1 
column, and in the T column the .138 is the difference between .496 
and .358 in the JT and 7” columns of row V.. The D row should be 
checked at this point to see that the number in the T column is the 
algebraic sum of the row. 

Matrix B. The numbers obtained in row D should be divided by 
the first number in the row (.769 in this case) and checked in the 
usual manner by means of the T column. In this manner a 1 always 
appears in the upper left-hand corner of Matrix B, which will re- 
duce the arithmetic considerably when solving this matrix. Let the 
first m numbers as obtained from this division in the first row stand, 
but eliminate the last two, and substitute — 1 in the — 1 column and 
in column T enter the algebraic sum of the row as now constituted. 
This is the first row of Matrix B. 

The second row of Matrix B is obtained from row V; of Matrices 
A and A’ by subtracting the numbers in Matrix A’ from the corres- 
ponding numbers in Matrix A (in the sample .134— .070=.064 ; 
1 — .021 = .979, etc.). Then enter — 1 in the — 1 column, and in the 
V, column the number from row V;, column V, of Matrix A (.146 in 
the sample). Check as usual by addition. 

. Build each of the n rows of Matrix B in the same manner, using 
successive rows of Matrices A and A’. The V, column of Matrix B 
(except in the first row) contains the same numbers in the same 
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order as the V, column of Matrix A (except the first two). The last 
row of Matrix B has a 0 in the — 1 column the same as Matrix A. 

It will be noted that in Matrix B there is a symmetrical square 
of n — 2 arrays, starting in the second row and column of the matrix. 
This reduces the number of subtractions necessary, since some num- 
bers can be copied into their proper places, as soon as one sees where 
the symmetry lies. (In the example these numbers in Matrix B are 
.064 , —.176, and —.119) 

Matrix B’. When the n rows of Matrix B are completed and 
checked, make a table of products in Matrix B’, exactly as was done 

®for Matrix A . Check the algebraic sum of each row as before. There 
will be a symmetrical submatrix of »—2 arrays contained within 
Matrix B’. (In the example the only symmetrical number is — .015 
since there are only four independent variables in the problem.) If 
this square is not symmetrical, an error has been made and must be 
corrected before proceeding further. (The error may be in the first 
row of Matrix B , where all the numbers were divided by a constant.) 

Row D’ . The differences below Matrix B, in row D’ are obtained 
from the second row of Matrices B and B’ in the same way that row 
D was obtained from row V, of Matrices A and A’. (In the sample 
.979 — .005= .974; —.119— (—.015) =—.104, etc.). The usual 
check by addition should be made. 

Matrix C. Build Matrix C from Matrix B exactly as Matrix B 
was obtained from Matrix A . There will be » — 1 rows in Matrix C. 

Matrix C’ . Matrix C’ is derived from Matrix C exactly as B’ was 
derived from B and A’ from A. 

Continue building matrices until the last contains but 2 rows. 
(This will be the nth matrix). 

From this last matrix the beta values are obtained. First, a D 
row is obtained by subtracting the products obtained at the right 
from the numbers in the second row of the matrix. In the sample 
these differences are .331, .124, etc., in row D”. The number thus 
obtained in the V, column (.331) is the first beta. The second beta 
is obtained by dividing the number at the foot of the V. column (.124) 
by the number that row 1 of Matrix B was divided by (.124 + .769 = 
.161). (This divisor is 1 — 7? ): The third beta is obtained by dividing 
the number at the foot of the V; column (.271) by the number that 
row 1 of Matrix C was divided by (.974). The fourth beta is simi- 
larly obtained by dividing the number at the foot of the V, column 
by the number that row 1 of Matrix D was divided by (.315 + .935= 
.337). For problems containing more variables, continue dividing the 
numbers obtained from the last matrix by the numbers that the top 
rows of each corresponding matrix were divided by. 
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Multiply each beta value by the respective correlation with the 
criterion, in the order entered in the C row of Matrix A . These prod- 
ucts should be added; the square root of the sum is multiple R. 

A sample five-variable problem is solved in Table 1. In Table 2 
is presented the proof of the beta values for this example, obtained 
by substitution in the standard formula quoted in the table. It is seen 
that the totals agree closely with the coefficients of correlation of the 
independent variables with the criterion. 
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DYNAMICS OF THE CEREBRAL CORTEX AUTOMATIC 
DEVELOPMENT OF EQUILIBRIUM IN 
SELF-ORGANIZING SYSTEMS 


W. R. ASHBY 


DEPARTMENT OF RESEARCH, BARNWOOD HOUSE 
GLOUCESTER, ENGLAND 


The nervous system, and particularly the cerebral cortex, is ex- 
amined in certain of its activities, it being treated as a purely physi- 
cal dynamic system. It is shown mathematically that if, as seems 
likely, the cerebral cortex can undergo manifold changes of neu- 
ronic organization, then it follows that these changes must auto- 
matically lead to the development of more and more states of equilib- 
rium, the process being unavoidable and largely irreversible. The 
theorem may be established on basic physical Sage ner without ap- 
peal to special physiological details. The possible significance of this 
observation is inilicated. 


1. 


It is generally accepted that the nervous system may be regarded 
as a purely physico-chemical or mechanistic system. This involves 
two assumptions: 

(a) That its state may be specified by the numerical values at 
any moment of suitably selected variables. (The technical difficulty 
of actual measurement and the great number of them required do not 
affect the printiple) ; 

(b) That if we knew the state of every reacting part we could 
predict with certainty what it would do next; or in other words, its 
change at any moment depends only on its present state. (It will 
commonly be necessary to include environmental actions and reac- 
tions to make the system complete.) 

This means (Note 1)* that the behavior of the system can be 
described by equations of the type 

ee Selassie (t=1,2,---,m). (1) 

dt 
Such a system of variables, i.e., whose fluxions are functions only of 
the variables of that system, ¢ in particular being absent from the 


* To avoid interrupting the line of argument with minor, but necessary, mat- 
ters, these have been collected into “Notes” at the end of the paper. 
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right-hand side, is an “absolute” system. It is a well-known theorem 
that such a system of equations defines a congruence of curves in the 
corresponding n-dimensional space having co-ordinates 2, , 2 ,+*+, Xn. 
This congruence, the “x-field,’ forms a representation of the beha- 
vior of the system which is very convenient for the later: proofs (Note 
2). 

The “organization” of the system must now be identified with 
the set of f’s of equations (1). No other identification is possible. 

It appears probable that some parts of the nervous system, par- 
ticularly the cerebral cortex, can undergo internal reorganizations 
which result in a change of behavior. The many facts of learning 
and training indicate this. In view of the known variability of beha- 
vior in the higher animals, it seems that this postulate must be al- 
lowed ; for if we hold to the “mechanistic” hypothesis we must allow 
that changed behavior must be the result of some inner, physical 
change (Note 3). 

These “reorganizations” are the subject of this paper. It will be 
shown that they must always tend in the direction of increasing equi- 
librium. 


2. 


It will now be shown that a spontaneous change of organization 
implies the presence of a step-function of the time. A change of or- 
ganization, by definition (§ 1), means that the functional forms f; 
must change to ¢;, say. This change may be represented with equal 
generality as being due to a parameter h, in the f’s, the h changing 
value from, say, h’ to h”. This gives the two required organizations, 


i.e., ™ 
dx; F 
ag Tt Rass Zn; h’) 

and 2 
ie. (2) 


a: Xoy°**s a3 h"). 
This change (from h’ to h”), however, by the “absoluteness” hy- 
pothesis (equation 1), is not to be arbitrary but is to depend on the 
internal events of the system. This means that h is to be treated as 
an extra variable in the system and that its change must be a func- 
tion of given x,h combinations. h, therefore, regarded as an ordi- 
nary variable of the system, will be observed in its time-development 
to be constant at h’ for a time, and then to change to h”, where it 
continues. h must therefore be a step-function of the time (Note 4). 
At this point it must be noted that the system composed of both 
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the z’s and h is absolute and has one field in +1 dimensions; and 
it has one and only one organization. This system must be clearly 
distinguished from the system composed of the x’s only, which is 
also absolute in the intervals while h is constant, and in this interval 
it has one field in nm dimensions. But owing to the possibility of 
h-change, the x-system has two possible organizations. The effect of 
the change of h is to change one xz-field to another (Equations 2) 
(Note 5). This sudden change of a field from one form to another 
due to movement of the x-point in the field is fundamental here, and 
the concept will be used repeatedly in the paper. It is useless to pro- 
ceed unless the reader is satisfied with the validity of this concept. 

The presence of more than one step-function in the system adds 
further possible spontaneous changes of organization in the x-system. 
If there are s step-functions h , each of which may take r values, then 
the x’s will have r* organizations. (Each step-function must have de- 
fined the x and h points at which it changes value. These points will 
be referred to as 6-points, from the equation in Note 4). 

We now define a commutive system as one which (a) is absolute, 
(6) contains step-functions of the time h;, (y) contains an infinite 
number of organizations of the variables x, provided by the step- 
functions h. 

It is now easily shown that the initial hypotheses of §1 are 
equivalent to: The nervcus system and its environment form a com- 
mutive system (Note 6). 


3. 


We now proceed to study a peculiar property of commutive sys- 
tems. A commutive system being absolute, its variables x; and h; 
will change with the time, the h’s change being discontinuous. (Some 
further properties are mentioned in Note 7). We now ask: Under 
what conditions can the h’s become permanently constant, in spite 
of repeated disturbances affecting the x’s? (Note 8) Constancy of 
the h’s is important since it corresponds to constancy of the 2-field ; 
and it is therefore the necessary and sufficient condition that the x’s 
should demonstrate a constant pattern of behavior after disturbance. 
An important sufficient condition is given by the following theorem: 
In any commutive system, if some external disturbance brings the 
x-point repeatedly to points A, B,--- (at random times and in ran- 
dom order) a sufficient condition (Note 9) for subsequent permanent 
constancy of the h’s is that an x-field should occur with both the 
x-point at that moment and the points A, B, --- all on paths which 
(a) after some finite time meet no new z-points and (b) meet no 





ee 
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6-points. Proof: On the occurrence of such a field, the zx-point may 
continue to move as long as we please (Note 10) but no 6-points can 
be met by the z-point, since it can only follow the paths from A, B, 
--- and these contain no 6-points. And as change of field occurs only 
on the x-point meeting a 6- point, change of field and of path cannot 
occur; so the x-point is permanently kept to these paths and is there- 
fore permanently barred from meeting a 6-point. So the h’s and the 
field are permanently constant. 

A definition of “equilibrium” is now required. A path is defined 
here as “equilibrial” if, from some point onwards, it always remains 
within a fixed region of the x-space (Note 11). It is clear that the 
paths mentioned above (which meet no new xz-points after a finite 
time) are equilibrial in the sense given. For after the finite time, 
the path must either come to an end, or join back on to itself form- 
ing a cycle; but in either case it remains within a finite region. 

As there are an infinite number of fields we now proceed by using 
probability methods (Note 7). It will now be shown that: if there is 
a constant probability p that a field provided by a random combina- 
tion of h-values in a commutive system will be of the type proved 
sufficient for -constancy above (Note 12), then the probability P 
that the system after m field-changes will be h-constant is at least 
(Note 13) 

1— (1—p)*. 


Proof: The occurrence of such a field being sufficient to cause h-con- 
stancy (here called “success”), and since a success stays so, while 
the failures at each field-change are given a fresh chance p of suc- 
ceeding, we have (if 1 — p= q) that at the first trial p succeed and q 
fail; in the second trial the q will be divided so that gp succeed and 
qq fail; and so on. The proportion succeeding at the m-th trial will 
be pq”, and P will be at least 


p+ pq + pq? +---+pqr?=1—(1—p)”. (3) 


In other words, m field-changes in a commutive system increase the 
probability of equilibrium from p to 1 — (1 — p)™, the latter always 
being greater. If we now add the trifling postulate that so long as the 
h’s are not constant m > © as t > ©, then clearly 

Lim P= 1. (4) 


too 


4. 


Equation (4) is the point of the whole paper. It means that, in 
a commutive system, however small p may be, making m (or ¢) large 
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enough will ensure that P approaches 1. In other words, no matter 
how rare equilibrial paths are in the individual x-fields, a commutive 
system will inevitably keep changing them until it does develop an 
equilibrium (Note 14). A commutive system is selective for organi- 
zations having equilibrium. And by our previous conclusion (that 
the nervous system and its environment form a commutive system) 
this must apply equally to the nervous system. 

We see therefore that the assumption that the nervous system is 
a physical system and that it can undergo changes of organization 
leads without any further hypothesis (Note 15) to the deduction that 
it‘must be a commutive system (§ 2), and this leads to the conclu- 
sion (§3) that it must tend to develop an organization giving equi- 
librium, no other organization being capable of persisting. 


5. 


This paper is hardly concerned with applications, but two may 
be mentioned. 

After the establishment of equilibrium has occurred, all disturb- 
ances (i.e., displacements towards A , B, ---) will be followed by ac- 
tivities of the system which will always oppose the disturbance (this 
being a characteristic of all systems in equilibrium). 

This opposition to disturbance may, if the activity of the reac- 
tion is small, explain the phenomenon of “habituation.” 

If the reaction to the disturbance is more active and complex, 
it may explain the development of “adaptive” behavior, since adap- 
tation seems to be essentially the development of that organization 
which will preserve equilibrium (Notes 16, 17, 18, 19). 


NOTES 


(1) This can be proved in much finer detail, buc the details are hardly re- 
quired here. Jts derivation from (b) is almost direct. 

(2) A configuration of the system corresponds to a point in the space. The 
behavior of the system in time corresponds to the point moving along a 
path. All the paths make the field and this defines the organization of the 
system. “Experimental control” is equivalent to the ability to start the 
point where we like in the field (by using the x arbitrary constants in the 
solution of equation 1). 

(3) We do not need here to know anything of the physiological mechanism un- 
derlying such change. 

(4) To preserve the form of equations (1) we may use any continuous ap- 
proximation to step-function form such as 


dh h’ 4. h” + a ae 
a 2 2 





tanh{q 9(x,,--- 12450} —h] ; 


where q is positive and large. The function 6 determines those x and h 
combinations where.h’ changes to h”; for where @(x,,°-: , &,; 2) is posi- 
tive, h becomes h’, while where it is negative h changes to h”. 
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(5) 


(6) 


(7) 


(8) 


(10) 
(11) 


(12) 
(13) 


(14) 


(15) 


(16) 


(17) 


(18) 


(19) 
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The whole z-field remains constant until h changes; then the «-field 
changes, in general to something quite different. Consequently, if the «- 
point, started from x,°, x,°, -:: , £,° always followed a particular path, 
then after change of h it can follow a different path even though started 
from the same (x) point. seeie. 
(a) The hypothesis (b) demands absoluteness. (8) The “reorganization” 
hypothesis demands step-functions, as we have shown above. (7) As we 
are not concerned with the possibility that the nervous system is limited 
in its variability of behavior, we may assume this to be infinite in the ab- 
sence of any reason to the contrary. . 
Since a commutive system has an infinite number of 2-fields, it clearly can- 
not be handled with the explicitness of equations (1). Assuming we have 
in front of us a real, physical example of a commutive system, we can 
clearly no longer know (a) the number of h’s, (b) their values, (c) the 
forms of the f’s in f;(x, ,-:: ; h,,°:*), (d) the «-fields, since these de- 
pend on the f’s. But, and this is sufficient for the rest of the paper, (a) 
we can observe the values and changes of the x’s, (b) we can control the 
x’s starting point, and (c) we can test for constancy of the h’s by seeing 
whether the «’s path in the x-space repeats itself. mid 
The disturbances are needed to test for, and demonstrate, equilibrium. 
It is not necessary, for there are other ways of getting h-constancy, de- 
pending on some relation between the distribution of 6-points and the di- 
rection of paths in the ficlds. We have no right to say that these combina- 
tions cannot occur. 
Some minor postulate that the disturbance is to return the x-point to A, 
B,.--- by some definite route devoid of 6-points is required here. 
Firstly, by selecting special examples it is easy to show that “equilibrium” 
belongs strictly to a single path and not to a field. Other special cases 
show that the path is to be confined to a region and not necessarily to 
terminate at a point. It may easily be shown that the common examples of 
equilibrium are all special cases, or limits, of the definition given. Nothing 
less general seems to be adequate. 
The x-point is first to be fixed, but p is not to depend on where it is fixed. 
“At least” because, as shown in Note 9, other ways of getting h-constancy 
are possible, though rare and of little importance. p does not include these, 
so their presence may result in an increase of the proportion which have 
become h-constant. 
This does not apply to a single field, i.e., the “commutive” part is neces- 
sary; for in a single field, as t > ©, P doesnot > 1 but —> p. If p is 
small the effect is quite different. 
The hypotheses about p and m are really only that (1) p ~ 0 and (2) 
m— © ast — ©. These merely exclude peculiar cases. 


If the 9-points are distributed systematically, the paths of equilibrium 
will automatically have systematic properties imposed on them. is leads 
to interesting and important developments which have already been ex- 
plored by the author, but they are outside the scope of this paper. 

The above discussion deals with one system where the «x-point either is, or 
is not, on an equilibrial path, and the equilibrium is therefore “all or none.” 
But there are several ways in which we can get independent equilibria, 
and then, following the same principles as before, the number of equilibria 
must tend to increase by accumulation. But this leads beyond the scope 
of the present paper. 


We have supposed the environment to remain constant as far as its or- 
ganization is concerned (though not the values of its variables). This 
means that the environment can interact freely with the nervous system 
in the above theory. A single change of the environmental organization 
would, however, wreck an established equilibrium. But regular changes 
between a finite set of environments can be shown to tend to equilibrium. 
If the environment should change its organization irregularly, the whole 
paper becomes inapplicable since postulate (b)-of § 1 is no longer true. 
Ashby, W. R. J. ment. Sci., 1940, 86, 478. 
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In the absence of sound units of measurement, the data ob- 
tained by use of a psychometric instrument may sometimes be so 
irregular that they cannot be accepted as a good representation of 
the true function of the trait measured. Frequently, ordered cate- 
gorical or ranked data and other irregular distributions are obtained 
to represent traits that are obviously continuous and probably un- 
skewed functions. In such cases, the data may be improved in the 
direction of a better approximation of the true function by convert- 
ing them to normal distribution form. A table of mean deviates of 
various portions of the unit normal curve is presented to facilitate 
this conversion. 


1. Normalized Data 

The practice of normalizing a body of data is not unusual in 
psychometrics. In the absence of sound measuring instruments, the 
ordered categorical or ranked data obtained to represent certain func- 
tions are frequently converted to a continuous scale and distrubution 
form by the use of the normal curve. A less frequent but equally jus- 
tifiable practice involves the conversion of continuous but distorted 
data so that certain corrections are applied to both the scale and the 
form of the distribution. 

Unfortunately, there is a tendency to believe that, when nor- 
malizing a body of data, one must be prepared to assert and justify 
the assumption that the true function represented by the data is a 
normal curve. Actually, the assumption of a normal curve function 
is unnecessary. In the absence of sound units of measurement and 
in the absence of knowledge of the exact mathematicah ‘function that 
represents the true distribution of a mental trait, any obtained psy- 
chometric data must (1) be accepted as a sound representation of 
the scale and distribution of the trait, or (2) be adjusted to fit a more 
probable scale and distribution. If a measuring instrument is sound 
and the distribution obtained is reasonable, the data may be accepted 
as a satisfactory representation of the mental trait. On the other 
hand, if a measuring instrument is not entirely sound and the dis- 
tribution obtained is not reasonable, some corrections may be applied 
to improve the scale and distribution form. In the case that the cor- 


* The author wishes to acknowledge the kind counsel of Professor Truman L. 
Kelley in the preparation of this article. 
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rection applied involves normalizing the data, it is not necessary to 
hold that the true distribution is the normal curve; the action may 
be taken soundly on the consideration that the normalized data pro- 
vide a better approximation of the true function than do the original 
data. 

It is possible that curves other than the normal curve might serve 
the corrective purpose in providing a better approximation of the 
true situation; the normal distribution, however, is quite reasonable 
in most situations, and it has the advantage of being a very conveni- 
ent practical curve. Its use to adjust certain data is justified if (1) 
the obtained data cannot be accepted as representing either the scale 
or the distribution form of the true function and (2) the scale and 
distribution form of the normal curve are considered a better approxi- 
mation of the true function than the obtained data. When these two 
conditions exist, the data may be normalized, and, unless a curve more 
appropriate than the normal curve is known, the corrections made 
can be considered to yield the best approximation of the true data. 

Fundamentally, therefore, a body of data is normalized to ob- 
tain better estimates of true values for the obtained measurements. 
This converted distribution, however, has other practical advantages. 
The normalized data possess a sound equal-unit linear scale, and the 
original defective distribution is modified in the direction of all the 
essential characteristics of the normal curve, a distribution wherein 
all the algebraic relationships are simple, very tractable,.and well 
known. 


2. Tables of Deviates 


A number of tables are available to assist in the conversion of 
data to normal distribution form. Nearly all of these tables, how- 
ever, require the use of median deviates for various portions of the 
curve. When a large number of curves are treated, the computation 
of accumulative proportions for the median of each class of the data 
to which a deviate is to be assigned, is awkward and laborious. Fur- 
thermore, the mean deviate may be considered the more appropriate 
statistic for the purpose. 

A table of mean deviates published by Thorndike (3) is not sat- 
isfactory in several respects. It provides deviate values for only one 
half of the curve, and all values are given to the second decimal only. 
In order to obtain the mean deviate for any portion extending over 
the .50 division of the curve, it is necessary to look up two values 
and compute a weighted average. This procedure is not only ineffici- 
ent, but the computation of a weighted mean is hazardous in view of 
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the two-decimal accuracy of the table values. For the same reason, 
it is inadvisable to make any interpolations. 

The table of mean deviates for various portions of the unit nor- 
mal curve presented here provides values accurate to the fourth deci- 
mal, and the deviates for each .01 portion or multiple thereof can be 
read directly from the table. In most cases, the hundredths propor- 
tions of the data to be normalized will yield sufficient accuracy in the 
deviates, but cautious linear interpolation is possible if more precise 
values are needed. 

The mean deviate values given in Thorndike’s table agree gen- 
erally with those of the new table. It should be noted, however, that 
Thorndike’s deviates for the first .01 to .10 portions of the curve are 
too large. 
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3. Table of Mean Deviates 

Figure 1 shows the variables used to compute the mean devia- 
tions for various portions of the unit normal curve and the arguments 
needed to obtain mean deviates from the table. In this figure, P rep- 
resents any portion of the unit normal curve for which the mean devi- 
ate Sp is desired. z, and 2, are the ordinates limiting P; q, and q2 are 
the proportions of the curve to the right of z, and z., respectively. 
The small letter p is used to designate the portion of the curve that 
is either above or below P. Subscripts a and b differentiate between 
p,., the portion of the curve above P including the positive tail, and 
p», the portion of the curve below P including the negative tail. The 
relationships among these variables to be noted are (1) that ¢: — q. 
=P and (2) thatp, + P+p=1. 
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The mean deviates given in Table 1 were computed in accord- 
ance with the formula given by Kelley (1). This formula may be 
written ¢ 

@1 — &2 
Sp — ° 
Q1 — Qe 
In special cases where P, the portion of the curve for which the mean 
deviate is desired, extends to and includes a tail of the distribution, 
this equation becomes 





—r. 
Sp P . 

The ordinate values, z, and z., used in computing mean deviates 
were taken from The Kelley Statistical Tables (2). To assure ac- 
curacy of the table, the mean deviates were computed twice. Since all 
numbers were carried to six decimal places and the deviations were 
rounded to four decimal places in accordance with standard practice, 
it is reasonable to assume that the table is correct to the fourth deci- 
mal place. 

The columns of Table 1 are determined by p values from .00 to 
.50; the rows, by P values from .01 to the limit where p + 4P = .50 
and the deviation of P is zero. Taking p from the positive end of the 
distribution (p above P, p.), the required mean deviate of P may be 
read directly from the table with a positive sign if p, + 4P < .50. 
When p, + 4P > .50, it is necessary to enter the table with the p value 
for the negative end of the distribution (p below P, p,) and the devia- 
tion of P thus obtained has a negative sign. As a universal rule, there- 
fore, the sign of a deviate read from Table 1 is that of the tail of the 
distribution included in p. 


4. Use of Table 1 
Table 1 facilitates the conversion of any distribution to normal 
curve form. For demonstration purposes, we shall normalize a dis- 
tribution in which the frequencies of successive categories are 25, 50, 
150, 75, and 200. The conversion is shown in Table 2, and the steps 
in the process are as follows: 


1. The categories are entered in the worksheet in positive to 
negative order as shown in column A. 

2. The frequency in each category and the total, N , are entered 
in column B. 

3. The frequency in each category is divided by N to obtain the 
P values in column C. 
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4. The proportion of the distribution above each category, 7. , is 
obtained by summing P values. In the example shown, p, is 
obtained for categories 1 through 3. Since for category 4, 
Dp. + 4 P >. 450, the procedure is reversed and 7 , the propor- 
tion of the distribution below each category, is obtained for 
categories 4 and 5. These p values are entered in column D. 

5. The mean deviation of each category is ubtained by entering 
Table 1 with the p and P value for that category. These devi- 
ates are shown in column E. When pp, is used (categories 1-3) 
the deviates are positive; when y is used (categories 4-5) the 
deviates are negative. 
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TABLE 1 


Mean Deviates of Various Portions of the Unit Normal Curve 
p =the proportion of the curve above or below 
P =the portion for which the mean deviate is required 


p of distribution above or below P 


Fg .09 01 .02 .08 04 05 .06 07 .08 .09 


01 2.6652 2.1766 1.9624 1.8132 1.6962 1.5987 1.5145 1.4398 1.3725 1.3107 
02 2.4209 2.0695 1.8878 1.7549 1.6474 1.5566 1.4772 1.4062 1.38416 1.2822 
03 2.2681 1.9841 1.8239 1.7027 1.6081 1.5177 1.4423 1.3743 1.38125 1.2550 


04 2.1544 1.9121 1.7676 1.6556 1.5623 1.4814 1.4094 1.3442 1.2844 1.2289 
05 2.0627 1.8494 1.7170 1.6125 1.5243 1.4472 1.3783 1.3154 1.2576 1.2037 
.06 1.9854 1.7936 1.6708 1.5725 1.4887 1.4150 1.3486 1.2880 1.2318 1.1795 


07 1.9181 1.7431 1.6282 1.5851 1.4552 1.8843 1.3203 1.2616 1.2070 1.1560 
08 1.8583 1.6967 1.5885 1.4999 1.4288 1.8551 1.2932 1.2361 1.1831 1.1333 
09 1.8048 1.65388 1.5513 1.4666 1.3930 1.8271 1.2671 1.2116 1.1599 1.1112 


10 1.7550 1.6138 1.5162 1.4350 1.3640 1.3002 1.2419 1.1879 1.1373 1.0897 


11 1.7094 1.5763 1.4880 1.4049 1.3362 1.2743 1.2176 1.1648 1.1154 1.0688 
12 1.6670 °1.5408 1.4513 1.8760 1.3095 1.2493 1.1940 1.1425 1.0941 1.0484 
138 16273 1.5071 1.4211 1.8482 1.28387 1.2251 1.1711 1.1207 1.0733 1.0285 


14 1.5898 1.4750 1.3921 1.8215 1.2588 1.2016 1.1488 1.0995 1.0531 1.0090 
15 1.5544 1.4444 1.3642 1.2957 1.2346 1.1788 1.1272 1.0788 1.0332 .9899 
16 1.5207 1.4150 1.8374 1.2708 1.2112 1.1567 1.1061 1.0586 1.0138  .9712 


17 #1.4886 1.3868 1.3115 1.2466 1.1884 1.1850 1.0854 1.0389 .9948  .9528 
18 1.4578 1.3595 1.2864 1.2231 1.1662 1.1140 1.0653 1.0195 .9761 .9348 
19 1.4282 1.3332 1.2620 1.2003 1.1446 1.0934 1.0455 1.0005 .9578 .9170 


20 1.3998 1.3077 1.2384 1.1780 1.1235 1.0732 1.0262 .9819  .9398  .8996 


21 1.38724 1.2831 1.2154 1.1563 1.1029 1.0535 1.0073 .9636 .9221 .8824 
22 1.3459 1.2591 1.1980 1.1852 1.0827 1.0341 .9886 .9456 .9047 .8655 
23 1.8202 1.2357 1.1711 1.1144 1.0629 1.0152 .9704 .9279 .8875 .8488 


24 1.29538 1.2180 1.1498 1.0942 1.0485 .9965 .9524 .9105 .8706 .8323 
25 1.2711 1.1909 1.1289 1.0743 1.0245 .9782 .9847 .89384 .8539 .8161 
26 1.2476 1.1692 1.1085 1.0549 1.0058 .9602 .9173 .8765 .8375  .8000 


27 1.2246 1.1480 1.0885 1.0357 .9875 .9425 .9001 .8598  .8212 .7842 
28 1.2022 1.1273 1.0688 1.0170 .9694 .9250 .8832 .8433  .8052 .7685 
-29 1.1804 1.1070 1.0496 .9985 .9516 .9078  .8565 .8271 .7893 .75380 


30 1.1590 1.0871 1.0306 .9804 .9341 .8909 .8500 .8110 .7736 .7376 


31 1.13880 1.0676 1.0120 .9625 .9169 .8741 .8337 .7951 .7581 .7224 
82 1.1175 1.0484 .9937 .9449 .8998 .8576 .8176 .7794 .7427  .7074 
33 1.0974 1.0296 .9757, .9275 .8830 .8413 .8017 .7689 .7275 .6924 


34 1.0777 1.0110 .9579 .9104 .8664 .8251 .7859 .7485 .7124 .6776 
35 1.0583 9928 .9404 .8985 .8500 .8092 .7704 .73832 .6975  .6629 
-36 1.0392 9748 .92382 .8768 .8338 .7934 .7549 .7181 .6827 .6484 


37 1.0205 9570 .9061 .8603 .8178 .7777 .7396 .7081 .6679  .6339 
388 1.0020 9395 .8893 .8440 .8019 .7622 .7245 .6883  .65838 .6195 . 
39 .9838 9223 .8726 .8278 .7862 .7469 .7094 .67385 .6388  .6053 


40 .9659 9052 .8562 .8119 .7706 .7317 .6945 .6589 6244 .5911 


Al 9482 8884 .8399 .7960 .7552 .7166 .6797 .6443 .6101 .5770 
42 9307 8717 .8238 .7804 .7399 .7016 .6650 .6299 .5959  .5629 
43 .9135 8553 8079 «7649 «7247 ~— 6868 ~=— 6505 ~=— 6155 = «5817 ~— «5490 


44 .8964 8390 .7921 .7495 .7097 .6720 .6360 .6012 .5677 .5351 
45 .8796 8229 .7764 .73842 .6948 .6573 .6215 .5871 .5537 .5212 
46 8629 8069 .7609 .7191 .6799 .6428 .6072 .5729 .5397 .5074 


47 8464 -7910 .7455 .7040 .6652 .6283 .5930 .5589  .5258  .4987 
48 .8301 -7753 =.7303 Ss «66891 «= «6506 = «6139 Ss «5788 = «5449S s«.5120 = .4800 
49 .8139 -7598  .7151 .6748 .6360 .5996 .5647 .5809  .4982  .4663 


50 _.7979 7443 .7000_— 6659566215 ~— 5853 = 5506 = .5170 = 4845 .4527 
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TABLE 1 (continued) 

































































p= .00-.09 
P— .51-.99 
p of distribution above or below P 
P .00 01 .02 03 04 .05 .06 .07 .08 .09 
51  .7820 .7290 .6851 .6449 .6071 5711 .5366 .5082 4707 .4891 
52 .7662 .7188 .6702 .6803 .5928 .5570 .5226 .4894 4.4571 .4256 
538  .7506 6986 .6554 .6158 .5785 .5429 .5087 .4756 .4484 .4120 
54 .7351 68386 .6407 .6014 .5648 .5289  .4949 .4619 4298  .3985 
55 .7196 6687 .6261 .5870 .5501 .5149 .4810 .4482 4162 .3850 
56 .7048 6588 .6116 .5727 .5360 .5010 .4672 .4845 4026 .8714 
57 6891 6890 .5971 .5584 .5219 .4871 .45384 .4208 .3890 .3579 
58  .6740 62438 .5826 .5442 .5079 .47382 .4896 4071 .8754 .3444 
59 .6589 .6096 .5682 .5800 .4989 .4598 .4259 .3985 .38618 3309 
60 .6439 5950 .55389 .5159 .4799 .4454 4121 .8798  .38482 .3173 
61 6290 5805. .5896 . .5018 .4659 4816 .3984 .38661 .8346 .3038 
62 6141 5660 .5258 .4877 .4520 .4178 .3847 .8525 .8210 .2902 
68 5993 5515 .5111 .4736 .4881 .4039 8709 3388 8074 .2766 
64 .5846 5871 .4969 .4595 .4241 3901 .3571 .8251 .2987 .2629 
65 5698 5227 .4827 .4455 .4102 .8762 .3484 8118 .2800 .2492 
66 .5552 5083 .4685 .43814 .8962 .8624 38295 .2975 .2662 .2354 
67 5405 4940 4548 .4174 .8823 .8485 3157 .2887 .2524 .2216 
68 .5259 4796 .4401 .4083 .86838 .8346 .3018 .2699 .2885 .2077 
69 .5118 46538 .4259 .8892 .8543 .8206 .2879 .2559 .2246 .1938 
70 ~=.4967 4509 .4117 4.8751 .8408 .8066 .27389 .2420 .2106 1797 
71 4821 4366 .3975 .8610 .8262 .2926 .2599 .2279 .1965 1656 
72 ~=.4675 .4222 .88838 .8469 8121 .2785 2458 4.21388 .1824 1514 
78 4529 4078 .8690 .8326 .2979 .2643 .2316 .1996 .1681 1370 
.74  ~=.4888 3984 .8547 8184 .2887 .2501 .21738 .18538 .1587 .1225 
15 -4237 .3790 8403 3041 .2694 .23858 .20380 §=©.1708 .1892 1079 
-76 4090 8645 .8259 .2897 .2550 .2214 .1885 .1563 .1245 .09381 
17 ~=©.8948 3499 .8114 .2752 .2405 .2068 1739 .1416 .1097 .0782 
18 38796 83853 .2969 .2607 .2259 .1922 .1592 .1268 .0948 .0630 
19 .2648 .38206 .2822 -2460 = .2112 1774 = =.14438 1118 .0796 .0477 
80 §=.3500 3059 .2675 .2313 .1964 .1625 .1298 .0966 .0642 .0821 
81 .3350 2910 .2526 .2164 1815 .1475 .1141 .0812 .0486 0162 
82 .3200 2761 .2377 .2014 «.16638 .1822 .0987 .0656 .0327 
83 8049 2610 .2226 .1862 .1510 .1168 .0830 .0497 0165 
84 2897 2458 .2078 .1708 .1856 .1011 .0671 .0335 
85 .2748 .2805 .1919 .15538 .1198 .0851 .0509 .0169 
86 .2588 2150 .1763 .1895 .1089 .0689 .0344 
87 2482 1993 .1605 .1235 .0876 .0523 .0174 
88 .2273 1834 .1444 1072 .0710 .0354 
89 .2118 1672 .1281 .0906 .0540 .0180 
.90 1950 .1508 .1114 0786 .0866 
91 .1785 13841 .0943 .0561 .0186 
92 .1616 1170 .0769 .0381 
93 .1444 0994 .0588 .0195 
94 1267 .0814 .0402 
95 .1086 .0627 .0207 
96 .0898 .0431 
97 0701 .0224 
98 .0494 
99 .0269 
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TABLE 1 (continued) 


p of distribution above or below P 























































































































































P10 eS ee ee es s. Sw oe 
O1 1.2538 1.2004 1.1505 1.1032 1.0582 1.0153 .9742 .9346 8965 8597 
02 1.2271 1.1754 1.1268 1.0807 1.0368 :9948 9544 .9156 .8781 .8418 
03 1.2016 1.1514 1.1040 1.0589 1.0159 .9747 _.9851__—«.8969_——.8600_—.8243 
04 1.1770 1.1281 1.0818 1.0377 .9956 9552 .9162 .8787 8423 8071 
05 1.1532 1.1055 1.0603 1.0171 .9758 9861 .8978 .8608 8250 .7902 
06 1.1302 1.0836 1.0393 9970 .9564 9174 8797 8482 8079-7785 
07 1.1079 1.0623 1.0189 9774 .9375 .8991 .8619 .8260 7911  .7571 
08 1.0863 1.0416 .9990 .9582 .9190 .8811 .8445 .8090 .7746 .7410 
09 1.0652 1.0214 .9796 .9894 9008 8635 8274 —.79238_—.7583_——« 7251 
10 1.0446 1.0017 .9605 9210 8830 8462 8105 .7759 .7422 —.7094 
11 1.0246 9823 .9419 .9030 .8655 .8291 .7939 .7597 .7264 .6989 
12 10050 .9634 9236 .8853 .8482 8124 .7776 .7438 7108 6786 
13.9858 «944990578678 ~=—«.8313_—«.7959 7615 —.7280 «6953-6684 
14.9670 . 9267 .8880 8507 8146 .7796 .7456 .7124 .6801 .6485 
15 .9485  .9089 8707 8339 .7982 .7636 .7299 .6970 .6650 .6336 
16 9304 8913 8536 8172 .7820 7477 _~—=«7144 ~—«6818 6501 _—«.6190 
17 9126 8740 8368 .8009 .7660 .7321 .6990 .6668 .6353 .6044 
18 .8951 8570 8203 .7847 .7502 .7166 .6839 6519 .6207 5900 
19 8779  .8403 8040 .7688 .7346 .7013 «6689 _—«.6372_—«.6062_—.5 757 
20.8610 .8238 .7879 .7520 .7192 .6862 .6540 6226 5918 5616 
21 8443 4 ©.8075 «.7719-.7375 = «70896712 6393 6081 5775.5 AT5 
22 8278 .7914 .7562 .7221 6888 .6564 6247 .5938 .5634 5336 
23.8115 —.7755 £7407 ~—«.7068~—«6739 6417 ~—=«6108_—«5795 54935197 
24.1955 = £7598 = .7253 6918 = .6591 62725960 = .5654 5354 .5059 
25 1796 7443 .710i 6768 6444 6127 5817 5514 5215 4922 
26.7639 7290 6951-6621 «629959845676 — «5374 — 078.4786 
27.7484 «= «.7188 «68016474 «6154 58425586 = 5236 «4941 6B 
28 .7331 6987 .6654 6329 6011 .5701 5397 5098 .4805 4516 
29 .7179 6838 ~—.6507~—«.6184_— 5869 ~—.5561 5258 ~—.4961 ~—.4669 ~—.4382 
30.7028 6690 6362 6041 5728 .5422 5121 4825 4535 4248 
31 .6879 6544 6218 .5899 5588 5283 .4984 4690 .4401 4115 
32 6731 6398 .6074 5758 .5449 5145 4848 «4555-4267 = 3988 
33.6584 6254 59325618 53105009 ~=—.4712_— 442141848851 
34 6439 6111. 579154785172, «4872 «4577 4287 «4001-3719 
35 6294 5969 5651 5340 5085 «4737 «444341543869 «3588 
36.6151 _—«.5827 «551152024899 ~— «46024809 = 40218787 ~—8.457 
37 6009 .5687 .5372 .5065 4763 .4467 4176 .3889  .3606 3326 
38 5867 5547 5234 4928 4628 4333 4043 ©3757 «18474 8.195 
39 5726 ~—«.5408 = 5097 ~—«.4792 ~=—.4493 4199 39103625 3343 (3065 
40.5586 5270-4960 = 4657 4359 4066 ~—3778 «3493 «8212 _—.2985 
41 5447 5132 4824 4522 4225 3933. 3362 3082 2804 
42 .5308 4995 .4688 4887 4092 .3801 3514 3231 2951 2674 
_A3 5170 _—4858 «455342533958 3668 ~—-3382_— 31002820 2544 
44 5083 4722 .4418 4119 .3825 3536 3251 .2969 .2690 2414 
45 4896 4586 © 42833986 «3698 «3404 «8119 2888 «= 2559-2288 
A6 4759 —«4451— 41493852 «3560 =—.8272_— 2988 = 2707 ~=—.2429 2158 
AT 4623 4316 «401537193428 3140 2857 2576 += 2298 +~—«.2022 
48 .4487 4181 3881 3586 «3295 «3009 «2725 2445 «(2167 1891 
49 4352 «4047 ~—«3748 345381632877 ~—«.2594 23182036 ~—.1760 
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TABLE 1 (continued) 















































p= .10-.19 
P— .51-.79 
p of distribution above or below P 
P 10 11 12 18 14 15 16 RT 18 19 
5L .4082 3779 .8481 .8188 . .2898 .2613 .2880 .2050 .1772 .1497 
52 8947 8645 .83848 .8055 .2766 .2481 .2198 .1918 .1640 .1364 
53 .8818 2511 .8214 .2922 .26383 .2348 2066 .1786 .1508 .1231 
54 8678 8377 .8081 .2789 .2501 .2215 1988 .1658 .1874 .1098 
55 §©.8544 8248 2947 .2656 .2368 .2082 .1800 .1519 .1241 .0964 
56 .8409 8109 .2814 .2522 .2234 .1949 .1666 .13886 .1107 .0829 
67. 3275 .2975 .2680 .2889 .2100 .1815 .1582 .1251 .0972 .0693 
58  .38140 2841 .2546 .2254 .1966 .1681 .13897 .1116 .0836 .0557 
59 .8005 2706 .2411 .2120 = .1832 1546 .1262 .0980 .0699 .0419 
60 .2870 .2571 2276 =©=.1985 .1696 .1410 .1126 .0843 .0562 .0281 
61 .2785 .2486 .2141 .1850 .1561 .1274 0989 .0706 .0423 # .0141 
62 .2599 .2800 .2005 .1718 .1424 «+4.1187 .0851 .0567 .0283 
63 .2463 2164 .1869 .1577 .1287 .0999 .07138 .0427 .0142 
64 .2326 2027 .1782 .1489 .1149 .0860 .0573 .0286 
65 .2189 1890 .1594 .1801 .1010 .0720 .0482 .0144 
66 .2051 1752 = .1455 = .1161 .0869 .0579 .0289 
67 1913 16138 .1816 .1021 .0728 0486 .0145 
.68 1778 1473 .1175 .0880 .0586 0293 
69 .1633 138382 .10384 0787 .0442 0147 
-70 1492 1190 .0891 .0598 .0296 
71 .1850 1047 0747 .0447 .0149 
-72 1207 .09038 .0601 .0300 
73 .1062 0757 0454 .0151 
74 =.0916 .0610 .03805 
-75 .0769 .0461 .01538 
-76 §=.0619 .0309 
17 0468 0156 
78 =.0815 
‘79 =.0159 
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TABLE 1 (continued) 

























































































































































































B = .20-.29 
= .01-.50 
p of distribution above or below P 
P 20 21 22 .23 .24 .25 .26 27 28 .29 
01 .8239 .7893 .7554 .7225 .6903 .6589 .6281 .5977 .5681 .5389 
02 .8066 .1724 .7390 .7064 .6746 .6434 .6129 .5829 .5535 .5245 
03 .7895 1557  .7228 .6906 .6591 .6282 .5980 .5682 .5390 .5102 
04 =.7728 .7394 .7068 .6749 .6488 .6132 .5832 .5587 .5247 .4961 
05 ~=.7563 .7233 .6910 .6595 .6286 .5983 .5686 .53893 .5105 .4821 
06 .7406 7074 .6755 .6443 .6137 4.58386 .5541 5250 .4964 .4682 
07 = .7241 6917 .6601 .6292 .5989 .5691 .53898 .5109 .4825 .4545 
.08  .7083 6763 .6450 .6143 .5842 .5547 .6256 .4969 .4687 .4408 
.09 .6927 .6610 .€300 .5996 5697 .5404 -5115 4830 .4550 4272 
.10 .6773 6459 .6152 5850 .5554 5262 4975 4692 .4413 .4138 
11 .6621 .6310 .6005 5706 5411 .5122 4837 4555 4278 .4003 
12 .6471 .6162 -5860 .5563 .5270 .4983 .4699 .4419 .4143 .3870 
13 6822 6016 .5716 .5421 .5180 .4845 .4563 .4284 .4010 .3738 
14 .6175- .5871 0573 .5280 -4992 AT07 4427 4150 .38876 .0606 
15 .6029 .5728 5432 5141 4854 4571 .4292 .4017 0744 8475 
16 8.5885 5586 .5291 .5002 .4717 .4486 «4158 3884 3612 .3344 
ae 5742 5444 .5152 .4864 4581 .4301 .4025 3752 3481 .o214 
18 .5600 .5304 5014 .4728 .4446 4167 .3892 .3620 051 .3084 
19 .5459 5 165 4876 4592 4311 .4034 3760 3489 221 .2955 
20 83.5319 5027 .4740 .4457 .4177 4.8901 .8628 33858  .3091 .2826 
21 .6180 .4890 .4604 .43823 .4044 38769 .8498  .8228 .2962 .2697 
22 3=6.5042 4754 4469 .4189 .3912 .3638 .8367 .3099 .2833 .2569 
_.23 4905 4618 4335 4056 — 3780 3507 .8237 .2969 .2704 = .2441 
24 .4769 .4483 .4202 .3924 .8649 .8377 .8107 ~ .2840 2576 ~=«.2318 
25 ©.4634 4849 .4069 .8792 .8518 8247 .2978 .2712 2448 .2185 
.26 4499 4216 3936 .0660 087 3117 .2849 .2583 2320 .2058 
BT .4365 .4083 .0805 0029 8257 .2988 .2720 .2455 .2192 .1930 
28 ~=sC«.4233:1 8951 .8673 .8399 .8127 .2858 2592 .2327 .2064 .1803 
_.29  .4098 8819 .8542 .8269 .2998 .2730 .2463 .2199 .19387 .1676 
80.3966 3687 34123189 .2869 2601-2335 .2071_— «1809-1548 
31 ~=.8834 3556 .8281 .8009 .2740 .2472 .2207 .1943 .1681 .1421 
32 3702 8425 3151 .2880 .2611 .2344 .2079 ~=.1816 1554 ~=.1298 
03 0571 8295 .8022 .2751 .2482 .2216 1951 .1688 1426 .1165 
34 0440 .3165 -2892 .2622 .2354 .2087 .1823 .1560 1298 ~=.1087 
235 3310 3035 .2763 .2493 .2225 .1959 .1695 .1432 .1170 .0909 
_ 36 179 .2905 2633 .2364 = .2096 1831 .1566 .1303 .1042 .0780 
37 2049 2775 .2504 .2235 .1968 .1702 .1488 1175 .0913 .0652 
38 .2919 .2646 .2375 .2106 .1839 .1573 .1309 1046 .0784  .0522 
39.2789 2516224661977 «171014451180 0917-0655 _—.0392 
_-40 ; .2660° r _ 2387 __ 21 17 _ _.1848 ; 1581 .1316 1051 .0788 .0525 .0262 
Al .2530 .2257 .1987 1719 .1452 .1186 .0922 .0658  .0394 .0131 
42 2400 2128 .1858 .1589 .1822 .1056 0792 .0527 .0264 
43 2270 199817281460 11920926 = .0661_—«.0396 —.0132 
44 .2140 1868 .1598 .13830 .1062 .0796 .0530  .0265 
45 .2010 .1738 .1468 .1199 .0932 .0665 .03899 .0133 
46-1879 (16081337 _ 1068 __.0800_.0533_.0266 a 
AT 1749 1477 .1206 0987 .0669 .0401 .0134 
48 .1618 1346 .1075 .0805 .0536 .0268 
49 .1486 1214 0943 .0673 .0404 .0134 
50 .1855 1082 0811 .0540 .0270 
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TABLE 1 (continued) 
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p = .20-.29 
P= .51-.59 
p of distribution above or below P 
P .20 21 22 .23 .24 .25 .26 ZT 28 .29 
61. 1222 .0949 0677 .0406 .0135 
52 .1090 .0816 .0544 .0272 
53 .0956 .0682 .0409 .0186 
54 .0822 .0548 .0274 
55 .0688 0412 .0187 
56 .0552 .0276 
57 .0416 0138 
58  .0278 
59 .0140 
TABLE 1 (continued) 
p = .80-.39 
P= .01-.39 
p of distribution above or below P 
r mS ol 32 30 34 .00 .36 oT 38 39 
01 = .5101 4817 .4588 .4261 .8989  .8719 .38451 .3186 .2923 .2663 
02 .4959 .4678 .4400 .4125 .8854 .8585 .8819 .8055 .2794 .25384 
03 .4819 .4589 .4268 .8990 .8720 .8452 .5187 .2924 .2664 .2405 
04 .4679 4401 .4127 .8855 .8586 .3320 .8056 4.2794  .25385 .2276 
05 .4541 4265 .8992 .8721 .3454 .8189 .2926 .2665 .2406 .2149 
.06 4404 .4129 .2959 4.8588 .8822 .8058 .2796 .2586 .2278 .2021 
07 4268 8995 .8724 .8456 .8191 .2928 .2667 .2408 .2150 .1894 
.08 4133 8861 .8591 .8825 .3060 .2798 .2588 .2279 .2022 .1767 
.09 3998 .38728 .8460 .8194 .2981 .2669 .2410 .2152 .1895 .1640 
10 3865 3595 .8828 .8064 .2801 .2541 .2282 .2024 1769 ~# .1514 
11 $8782 0464 .8198 .2984 .2672 .2412 .2154 1897 .1642 .1388 
12 8600 000 ©.0068 ~—««2805 = 2544 = 2285) ~=—S 20027 1771 .1516 .1262 
13 .3469 3202 .2988 .2676 .2416 .2157 ~ # .1900 1644 .1389 .1186 
14 .88388 3072 .2809 .2548 .2288 .2080 3 4.1773 1518 .1268 # .1010 
45 6.8207 2948 .2680 .2419 .2160 .1903 .1647 1391 .11387 .0884 
16 .3078 2814 .2552 .2292 .2088 #=®%j1776 + #.1520 1265 .1011 .0758 
17 + .2948 .2685 .2424 .2164 .1906 .1650 .1394 .11389 .0885 .06382 
18 .2819 2557 .2296 .2087 4.1779 #+$.1528 .1268 .1018 .0759 .0506 
19 = .2691 .2429 .2169 .1910 .16538 .1897 .1141 .0887 .0633 .0380 
20° 2562 .2301 .2041 1788 .1526 # .1270 8 ©.1015 .0761 .0507 .0253 
21 .2484 2174 .1914 .1656 .1400 .1144 .0889 .0685 .0881 .0127 
22 2807 2046 .1787 .1580 8 8=©.1278 §©6.1018 .07638 .0508 .0254 
oo. aT 1919 .1661 .1403 .1147 .0891 .06386 .0881 .0127 
24 8.2052 1792 .1584 .1276 8 .1020 .0764 .0509 .0255 
25 1924 1665 .1407 .1150 .0898 06388 .03882 .0127 
.26 1797 1588 .1280 .1023 .0767 .0511 .0255 
27 ~=.1670 1411 .1153 .0896 .0640 .0884 .0128 
28 .1543 1284 .1026 .0769 .0512 .0256 
29 +1416 1157 .0899 .0642 .0885 .0128 
30 1288 1030 .0O771 .0514 .0257 
ol 1161 .0902 0644 .0386 .0129 
J 1033 0516 .0258 
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TABLE 1 (continued) 


= 40-.49 
Pp = .01-.19 


p of distribution above or below P 


P 40 Al 42 43 44 45 46 AT 48 49 


01 .2404 2147 .1891 .1637 1883 .1130 .0879 .0627 .0876 .0125 
02 .2276 2019 .1764 .1510 .1256 .1004 .0753 .0502  .0250 
03  .2148 1892 .1637 = .1383 §=6.11381_ 3.0879 )3=s 0627:~=—s 0876 = .0125 


04 .2020 1764 .1510 .1257 .1005 .0753 .0502 .0251 
05 .1892 1638 .1384 .11381 .0879  .0627 .0876 .0125 
06 .1765 1511 .1258 1005 .0753 .0502 .0251 


07 .1639 1385 .1132 .0880 .0628 .0877 .0126 
08 .1512 1259 .1006 8.0754 .. 0502 .0251 
09 .1386 1133 .0880 .0628 .0377 .0126 


10 # .1260 1007 .0755 .0503 .0251 


Al = 34 0881 .0629 .0877 .0126 
12 = .1008 0756 .0503 .0252 
13 .0882 0630 .0378 .0126 


14 0757, .0504 .0252 
15 .0631 0378  .0126 
16 .0505 0252 


17 ~=.0379 .0126 
18 .0253 
19 .0127 
































TABLE 2 
A B C D E 
Category N r p Sp 
Do 
2B 25 .05 .00 2.0627 
2. 50 10 05 1.3002 
3 150 30 15 5422 
Py 
4, 75 15 40 —.0631 
5. 200 40 .00 —.9659 
500 1.00 
REFERENCES 


1. Kelley, Truman L. Statistical method. New York: Macmillan, 1923. 
Kelley, Truman L. The Kelley statistical tables. New York: Macmillan, 1988. 


8. Thorndike, Edward L. Introduction to the theory of mental and social meas- 
urements. New York: Teachers College, Columbia University, 1918. 
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NOTE ON THE SAMPLING ERROR OF THE DIFFERENCE 
BETWEEN CORRELATED PROPORTIONS OR 
PERCENTAGES 


QUINN MCNEMAR 
STANFORD UNIVERSITY 


Two formulas are presented for judging the significance of the 
difference between correlated proportions. The chi square equiva- 
lent of one of the developed formulas is pointed out. 


It is well known that the sampling variance of the difference be- 
tween two proportions, p, and p., based on independent samples is 
given by 





og—o +0 en 4 (1) 
Pa N, Nz 
or by 
Pq pq 
2. => —, 2 
v= yD, (2) 


in which p, and p, as separate estimates of the unknown population 
parameters are replaced by 


te DpiN, + poNe 
N,+N, 


as a better estimate which is consistent with the to be tested null hy- 
pothesis, and therefore to be preferred. It is also known that when 
o*2 is calculated by (2) the value of d?/o*; equals y? with one degree 
of freedom. 

There are many situations in which the sampling variance of 
the difference between two proportions (or percentages) must take 
into account the fact that the two proportions are not based on inde- 
pendent samples. For example, p, may represent the proportion of 
individuals who give a particular response prior to a provided experi- 
ence and p. the proportion of the same individuals giving the response 
after the interpolated experience; or p, might be the proportion pass- 
ing one test item and p. the proportion of the same group passing a 
second item, and we wonder whether items differ significantly as to 
difficulty ; or we may wish to test the significance of the difference 
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between the responses of a group to two opinion questions; or we may 
need to evaluate the difference between two proportions based on 
paired or matched individuals. 

For such situations as the foregoing, neither formula (1) nor 
(2) is applicable. To be applicable, a subtractive correlational term 
should be included. The needed correlation is that between the sam- 
pling variations of the two proportions. Since it can readily be shown 
that proportions are means, and since it is known that the correlation 
between sample means equals that between variates, it follows that 
the needed correlation is that between first and second responses, or 
between performances on the two items being compared for difficulty, 
or between responses to two opinion questions, or between the re- 
sponses of matched cases. The correlation “scatter plot” for these 
situations involves a two-by-two table, for which one may apparently 
choose either the contingency coefficient or tetrachoric r or the four- 
fold point 7 as the measure of correlation. Which type of coefficient 
do we need for the sampling variance of the difference between corre- 
lated proportions? 

If the answer to this question is to be found in the statistical 
literature, the writer has failed to locate it. If the answer were obvi- 
ous, one would not expect the near universal failure of researchers 
to include the needed correlational term in those situations where it 
must certainly be taken into account in testing significance. We have 
found only one text* which considers the correlational term, and the 
authors of this text, after saying that “it is seldom that we know the 
correlation factor when dealing with proportions,” go on to state with- 
out proof that “it would need to be computed by the tetrachoric meth- 
od.” We shall presently see that this idea is incorrect. 

The desired sampling variance formula would be of the form 


og—o? +o*° —2r ec, (3) 
—” P2 MP2 Ti De : 


in which ¢ is not as yet specified. Suppose a fourfold table of frequen- 
cies, and the corresponding table of proportions, for a first (I) and a 
second (II) set of responses from the same individuals (see Table 1). 
The difference between the two marginal values, p. — p, , is the dif- 
ference for which we seek a standard error. For the sake of exposi- 
tion, let us assume that this difference represents a change in re- 
sponses. Then the B and C individuals are those who have not 
changed, while A represents those who moved from yes to no, and D 
those who changed from no to yes. The lower-case letters represent 
the corresponding proportions. Let us.score those who changed to 


* Peters, C. C. and Van Voorhis, W. R. Statistical procedures and their 
mathematical bases. New York: McGraw-Hill, 1940. P. 183. 
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TABLE 1 
Fourfold Table of Original and Proportional Frequencies 
(II) (II) 
No Yes No Yes 
Yes A B A+B Yes a b ae? 
(I) (I) 
No € OD | C+D No e 6uUd q 
A+C B+D N  » wo 


yes as +1, those who changed to no as —1, and those who changed 
not at all as O. This leads to the frequency distributions for changes 
shown in Table 2, which includes easily derived expressions for the 
mean and variance of the distributions of changes. 








TABLE 2 
Distributions for Changes in Terms of Original and Proportional Frequencies 
Original Proportional 
xX f Xx f 
+1 D 2 d 
0 B+C 0 b+e 
—1 A —l a 
N. 1.00 
Mean=(D—A)/N = d—a=p,—pD, 


sDi=— [N(D+A) — (D—A)2] = (d+a) — (d—a)? 


Obviously, the mean change equals (D — A)/N = (d — a) = 
P. — p~,, or the mean of the changes (or differences) equals the dif- 
ference between the two proportions. The sampling error of p. — p 
must be the same as that for M , its equivalent, and the sampling vari- 
ance of M is easily obtained by dividing the distribution, variance by 
N. Thus, in terms of proportional frequencies, we have \, 


oy =o, == [(d +0) — (d—a)4), (4) 


Now by easy, but somewhat cumbersome algebra which need not 
be reproduced here, it can be shown that (4) reduces to (3) providing 
the r of (3) is the fourfold point correlation coefficient. It will be 
noted that (4) requires less computation than (3) even when r has 
been calculated for its own sake. 
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It will be seen that (3), hence by implication that (4), involves 
the use of two separate estimates of the unknown population propor- 
tions, and therefore both are analogous to (1). If we wish a formula 
analogous to the preferred (2), that is, one involving an estimate 
based on the combined sets of responses or a p based on the total as a 
better estimate, we note that such a p would, as in the case of (2), 
imply that the population values for the two proportions are equal. 
This would be consistent with the null hypothesis that p. — p, = 0 
except for random sampling fluctuations. The proper p to replace p. 
and p, in (3) becomes the simple average of p. and p, since for the 
given situation N, = N, = N. Now the null hypothesis that p. = p, 
also implies that the equivalent, d — a, must equal zero except for 
sampling errors; hence formula (4) becomes 


SS ion 
f= y(dt+a), (5) 
which is algebraically identical to (3) with p. and p, replaced by p. 
Since (2) is associated exactly with y’, it is of interest to point out the 
connection of (5) with 7’. Table 3 contains the setup for 7? for the 
original observed (O) frequency distribution of Table 2. Since the net 





TABLE 3 
Schema for x? Test of Changes 
X 0 E (0—E)?2/E 
r D (D-+A)/2 (D—A)? 2 _1@—A)* 
= 7 4 Beh 8 Bis 
0 B+C 


(A—D)? 2 —1(D—A)? 
4 D+A 2 D+A 





—1 A (D+A)/2 


change (or difference) between the two sets of responses depends 
upon the D and A individuals, the expected frequencies (FE) are writ- 
ten on the assumption that the D + A cases would be divided evenly, 
on the basis of the null hypothesis, between plus and minus changes. 
The value of 7? becomes the sum of the right-hand terms, or 
_ (B-—-&)* 
ab +A 


When we write the square of the critical ratio with (5) as the 
sampling variance, we have 


(6) 


—_— 
a iia d+a 


N 
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Replacing proportions by frequencies, this becomes 





(5 A\? 

N N) _ (D—A)? 
DA era 
N N 

N 


Thus (CR)?, using formula (5), is identical to the y* of (6). Obvi- 
ously, CR = (D— A)/\VD + A, and therefore the significance of the 
difference between p. and p, can readily be computed from the four- 
fold table of frequencies or from the fourfold table of proportions. 

The use of either (5) or (6) is to be preferred over (3) or (4) 
just as, and for the same reason that, formula (2) is preferable to 
(1). 

Although the foregoing development was in terms of changes in 
responses, the resulting formulas are applicable to the other three sit- 
uations mentioned at the beginning of this note. The limitation on chi 
square regarding size of expected frequencies suggests that formulas 
(6) and (5) should not be used unless A + D is 10 or greater. 
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HENRY B. MANN. Nonparametric tests against trend. Econometrica, 1945, 13, 
245-259. 


This paper provides a statistical test of the hypothesis that a sequence of 
measures does not show a downward trend. The test against upward trend can 
be made in a completely analogous manner. The test is nonparametric in that no 
assumption is made about the distribution of the measurements. The test is based 
upon the rank order of measurements, X,, X,,--:, X, considered as a member 
of the class of permutations of the first n integers. Each of the permutations is 
considered equally probable. 

The first or T test is based upon the number of inversions occurring among 
the n measurements in the sample. An inversion is a situation where X; < X; and 
41<j.T7, the number of such inversions, is the sample statistic. The probability 
of obtaining a permutation of the first n integers such that T < T is tabled for 
T=0,1,---,21,andn=8,4,---,10. Forn > 10, the probability may be 
estimated from the normal curve. 

If the hypothesis of randomness is true, the probability of any X; being less 
than any X; when i < k is .50. The T test is shown to be consistent with respect 
to any of a wide class of alternative hypotheses when this probability is different 
from .50, less than .50 on the average, and when all measurements are independ-: 
ent. By consistent it is meant that the probability of rejecting the hypothesis of 
randomness when some other hypothesis among the prescribed alternatives is 
true, approaches certainty as the size of the sample increases without limit. 

A second test of the hypothesis is also presented. It is more powerful than the 
T test if the probability that X; > X, increases rapidly with j-i. In the sample, 
X,,X,,--:,X,, it may always be true that X; > X, if i<j and the difference in 
ranks is K or more. The smallest value of K for which that statement is true is 
the sample statistic. The probability of various values of K for any size sample 
under the hypothesis of randomness is tabled in the article. 

FELS INSTITUTE FOR RESEARCH IN HUMAN DEVELOPMENT 


ALFRED L. BALDWIN 


GERHARD TINTNER. Multiple regression for systems of equations. Economet- 

rica, 1946, 14, 5-36. 

Tintner in this publication faces the general problem of estimating the num- 
ber of independent relations among a group of economic variables, and further 
to estimate the coefficients in the following set of equations which describe these 
relationships. 

Dp 
ky + 2k, M;,,=0 v=—1,2,---,R t=1,2,---,N. 
j=1 
The R such equations represent the R independent linear relationships among 
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the variables. Each such equation, or set of equations, will relate the p variables, 
indicated by the j subscript, at the various times t = 1, 2, etc. To estimate the 
number, F , of independent relationships and the values of k,; , the sample contains 
values X;,, of the p variables at N times. The sample statistics needed for the 
statistical solution are the means and variances of each variable over the time 
series, and the covariance of every pair of variables over the time series. Since 
time series are being used, the estimation of the variance must include only the 
random portion of the temporal data. Tintner’s own variate difference method 
may be used to obtain estimates of the variance of each variable, although its 
use is not integral to the discussion. 

The statistical solution Tintner proposes closely resembles the factor analysis 
of a matrix of variances and covariances by the method of principal components. 
What is required is first a solution cf the determinantal equation | a;; —» V; | =0 
and then a substitution of the roots of this equation to obtain values of k,;, the 
structural coefficients. V; is the variance of each variable as determined by the 
method of variate differences or some similar method. 

Using a method developed by Hsii, Tintner, determines the rank of the matrix. 
He calculates all the principal components of the matrix by Hotelling’s method, 
and then uses the following test function: 


A,=(N—1) (A, +A +A, +°°°*A,)> 


where \, is the smallest root, », the next smallest, etc. This function is distributed 
as x2 with (VN —1— p+ 1) degrees of freedom if the rank of the matrix is r. 
By calculating successively \, , A,, --- until the hypothesis rank = r is rejected, 
the rank is estimated at R, the largest value of r for which the hypothesis is ac- 
ceptable. 

By substituting the RF smallest roots of |a;; — 4 V; | = 0 into the equation, 
and by further subjecting the coefficients to the conditions: 


Dp 
1. 2k,; k,,; Vi; —=5,~, where 5,,, is Kronecker delta, and 


w1 


N p Dp 
2 2 (2k; x5.) (ZK; H;,) =0, 
t=1 j=1 j=1 
R linear functions are obtained describing the FR linear relationships among the 
variables. 


At this point Tintner has mathematical expressions of the interrelations, but 
they are not very meaningful in terms of economics. To identify each relation- 
ship, he recalculates it under the assumption that some one or more of the coef- 
ficients in that relationship are equal to zero. This assumption is made on the 
basis of economic theory. For example, he assumes that in a “demand” curve for 
agricultural products, the prices paid by farmers has no weight, i.e., the coeffi- 
cient of that term in the equation is equal to zero. 

Making these assumptions generally reduces the fit of the curves to the 
data, but the reduction in fit can be tested. The ratio, A,/A’,, under different 
assumptions is approximately distributed as F if the fit under the different as- 
sumptions is equally good. Not only can the general fit be evaluated statistically, 
but each coefficient can be tested individually. The method is illustrated by a 
problem including five variables describing agricultural data. 

The method is suitable only for large samples at present. If the number of 
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variables were at all large, the computation would be excessively laborious, be- 
cause a complete factor analysis into principal components is necessary and a 
calculation of the inverse of the matrix of variances and covariances is required. 
On the other hand the existence of a statistical test for rank and for the coeffi- 
cients is highly desirable and might make the method worth the additional compu- 
tation——additional, that, is to that involved in some other method of factor anal- 
ysis. While the k,; are not factor loadings, some of the methods used could be 
adapted to the usual factor-analysis problem. 
ALFRED L. BALDWIN 
Fes INSTITUTE FOR RESEARCH IN HUMAN DEVELOPMENT 
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